Four days in Avignon: a papal fortress with Roman Provence half an hour away in every direction

Come in through the gate by the station and walk up the main street to the Place de l'Horloge, which has the town hall, the opera house and a carousel on it and is where Avignon does its sitting about. It is expensive and it is worth one drink for the setting; the squares two streets in either direction charge half as much.

Beyond it the town opens onto the Place du Palais and the palace fills the whole of one side. Inside are twenty five rooms of scoured stone: the Revolution used the building as a barracks and a prison and took the frescoes off the walls with them. What survives is worth the visit on its own, particularly the pope's private study painted with hunting and fishing scenes that have nothing religious about them at all, and the chapel big enough to hold a village.

Finish on the bridge. The Pont Saint-Bénezet was built in the 1180s, ran twenty two arches across to the far bank, and lost them to the Rhône so many times that in the seventeenth century the town gave up rebuilding it. Four arches and a chapel are left, sticking out over the water and stopping. The song says people danced on the bridge; the usual correction is that they danced under it, on the island.

Take an early train south. Arles is twenty minutes away, it was one of the great cities of Roman Gaul, and the ruins are not in a park somewhere: they are wedged into the middle of an ordinary working town, with houses built against them.

Start with the amphitheatre, which went up around AD 90, held twenty thousand people and was later turned into a fortified village with two hundred houses inside the ring, cleared out only in the nineteenth century. Three medieval towers on top are left over from that. It is still in use for bullfighting, in the local bloodless form where the man tries to take a rosette off the bull's horns and usually ends up over the barrier.

Then Saint-Trophime on the main square, whose twelfth century doorway is one of the great pieces of Romanesque carving in France, with a cloister off it that mixes Romanesque and Gothic in two pairs of galleries. Walk south afterwards to the Alyscamps, an avenue of Roman stone coffins under poplars that was the burial road out of the city for a thousand years. Van Gogh and Gauguin came out here together in the autumn of 1888 and both painted it, which is the closest thing Arles has to a quiet monument to the fifteen months he spent here.

The bus goes out through Remoulins in about three quarters of an hour and drops you at the visitor centre. The aqueduct is round the corner from it and the scale takes a moment: three tiers of arches, roughly fifty metres from the river to the top, built without mortar in blocks that weigh several tonnes each, carrying a channel that ran fifty kilometres from a spring to the taps of Roman Nîmes.

The engineering is the thing worth understanding, and the museum by the car park does it properly: the whole canal fell about seventeen metres over its length, an average of a few centimetres per hundred metres, surveyed by men with wooden instruments. Then walk across the lower deck, swim off the shingle below if it is hot, and take the path up the far bank for the view everybody has seen.

Come back for the middle of the afternoon and spend it in the Calvet, which is the town museum: five centuries of painting in an eighteenth century mansion with a courtyard of plane trees, free on the ground floor of most people's expectations and much better than it needs to be. Or skip it, buy a bottle from a shop and take the ferry over to the island for the evening.

The Collection Lambert is the surprise of Avignon: two eighteenth century mansions knocked together to hold a dealer's contemporary collection, all white rooms and cool floors, with Basquiat, Twombly and a great deal of video and photography. It is the opposite of everything else in the town and it is the museum people talk about afterwards.

Then the covered market on the square east of the centre, which has a wall of living plants across the front of it and, inside, the fish, cheese and vegetable stalls the restaurants buy from. It shuts at lunchtime and it is the best place to assemble something to eat on the train.

The shuttle to the TGV station runs from Avignon Centre, five minutes outside the wall, and takes a quarter of an hour. Do not leave it to the last train.