Twelve days in the Caucasus: three countries, three alphabets, one road

Start at the bottom of the old city, at the Maiden's Tower: a squat stone cylinder about thirty metres high with a buttress on one side, whose purpose nobody has settled after a century of archaeology. Fire beacon, fort, observatory and Zoroastrian temple have all been argued, and the dating runs from the seventh century to the twelfth. The roof gives you the bay.

Work uphill through the lanes to the Shirvanshahs' palace, the best medieval building in Azerbaijan and a compound rather than a single building: the fifteenth century palace, a domed audience hall in its courtyard, a mosque and minaret, the family burial vault, a bathhouse cut into the rock and a covered reservoir.

Then walk without a plan. Icherisheher covers twenty two hectares, went on the World Heritage list in 2000, and is still a residential quarter, so between the caravanserais and the carpet shops there are cats, washing lines and cars parked at improbable angles.

An hour south along the coast, on a hillside of tumbled limestone above the Caspian, are around six thousand petroglyphs cut into the rock over tens of thousands of years: cattle, boats full of rowers, hunters, and rows of long-limbed figures that look as though they are dancing. UNESCO listed the site in 2007 and the museum at the entrance is genuinely good. Take the guide.

One inscription is much later. A soldier of the Twelfth Legion cut his unit's name into a rock here in the first century AD, and it is the easternmost Roman inscription ever found.

A few kilometres further, a rough track leads to a field of mud volcanoes: low grey cones burping cold clay and gas, of which Azerbaijan has several hundred, something like a third of all of them on earth. Drivers run people out in battered Ladas because ordinary cars will not manage the track.

Stop at Bibi-Heybat on the way in. The thirteenth century shrine was blown up by the Bolsheviks in 1936 and rebuilt in the 1990s to the old plans, its striped domes standing over the derricks on the shore where oil drilling started in 1846.

Go out first to the Heydar Aliyev Center, which Zaha Hadid designed as one continuous surface peeling off the plaza and folding over into a roof, with no sharp angle anywhere. It opened in 2012 and took the Design Museum's design of the year two years later. Inside are galleries, a hall of Azerbaijani craft and the state car collection.

Back in the centre, the carpet museum occupies a building shaped like a half-rolled rug, which should be dreadful and is not. Several hundred carpets are arranged by weaving region with the looms, the dyes and an explanation of what the motifs mean, and it will change how you look at every rug for the rest of the trip, which matters when you have nine more days of carpet countries ahead.

Finish on the boulevard, the seafront park that runs for kilometres along the bay under plane trees, with tea houses and a small canal with boats on it, and the whole city walking on it after six.

The bus runs west across the dry Absheron scrub and then along the foot of the Greater Caucasus, with the range building on your right the whole way, until the country turns green somewhere past Shamakhi. Five hours.

Go straight to the Khansarai, because it shuts earlier than you expect. It is two storeys, small, and gives nothing away from outside. Inside, every surface is painted with hunts, flowers and battles, and the windows are shabaka: lattices assembled from thousands of pieces of hand-cut coloured glass and wood, fitted together without glue or a single nail. It was listed by UNESCO in 2019 along with the historic centre around it.

Then down to the caravanserai, a vast eighteenth century courtyard of brick with two galleried storeys, the biggest on the Caucasian stretch of the silk road and now partly a hotel. Walk into the courtyard whether you are staying or not and drink tea in it.

Take a car for the morning up to Ilisu, an hour northwest past Qax and then into a side valley. It was the seat of its own sultanate until the Russians ended that in the 1840s, and it is a village of stone houses under the high mountains with a defensive tower, an eighteenth century mosque and a gorge behind it. Most people here are Tsakhur, one of several small Caucasian peoples in these valleys with their own languages, and the drive up is half the reason to go.

Back in Sheki, the Juma mosque was built between 1745 and 1750, with a spire forty metres high and geometric brickwork, and the call to prayer off it carries down the whole valley.

Spend what is left of the day in the tea houses and the bazaar, which is the thing Sheki is actually for. Tomorrow is a border.

The Lagodekhi crossing is three hours from Sheki and the whole run to Tbilisi takes four to five, most of it through the Kakheti vineyards on the Georgian side. Take a shared taxi rather than a minibus if you can: it will wait for you at the border, and the formalities are quick but not instant.

Drop your bag and walk the lanes climbing from the river towards Freedom Square. This is the part of Tbilisi that survived everything: crooked houses round courtyards, carved wooden balconies over the pavement, churches wedged between apartment blocks, an abandoned tram left where it stopped.

Then down to Abanotubani, where the bathhouses are dug into the slope so only their brick domes show above ground and the water arrives from the spring beneath at around forty degrees. Entry to the public bath costs almost nothing and a private room by the hour costs a little. Take a towel and something for your feet, and pay for the scrub, which is administered rather than offered.

Cross to the left bank early for Metekhi, the small church on the rock directly above the water. There has been a church and a royal residence here since the fifth century; the present building is late thirteenth century, and the terrace outside lines up the baths and the fortress across the river.

From Rike Park below, the cable car crosses to Narikala in about two minutes for a couple of lari. The fortress has held the ridge since the fourth century, most of what stands is far later, and there are no railings, no ticket and no queue.

Walk the crest west to Mother Georgia, twenty metres of aluminium put up in 1958 for the city's fifteen hundredth birthday, holding a bowl of wine for guests and a sword for everybody else. Walk down through Sololaki into dinner, and order less than you intend to.

Mtskheta was the capital of Iberia for the better part of a thousand years and sits twenty minutes north where the Aragvi meets the Mtkvari. Marshrutkas leave Didube constantly and cost a couple of lari.

Jvari first, on the hill opposite, built around the year 600 over the spot where St Nino, the woman who converted the country, is supposed to have raised a wooden cross. It is the building every Georgian church copied for the next five hundred years: a compact cross plan, bare stone, no decoration, perfect proportions, with the two rivers meeting below.

Down in the town, Svetitskhoveli was founded in 1010 on the site of Georgia's first church, with kings buried under the floor and a pillar in the middle said to stand over the robe of Christ. Samtavro, ten minutes on, is a working convent holding the graves of King Mirian and Queen Nana, the pair who made the country Christian in the fourth century.

Back in Tbilisi, buy your Yerevan seats for tomorrow. Shared taxis and minibuses leave through the morning and the good ones fill.

Six hours over the Bagratashen crossing, through the Debed canyon on the Armenian side with monasteries visible up on the slopes, and then out onto the bare plateau. Shared taxis are worth the extra over the minibus on this leg.

Arrive and start where the city starts. Republic Square is an oval rather than a rectangle, ringed by five monumental buildings from the 1930s to the 1950s, all faced in the pink and cream tuff Tamanyan insisted on, with the paving laid out in the pattern of an Armenian carpet.

Walk north to the Opera and on to the Cascade: a stairway of limestone terraces climbing the hill behind the centre, begun in the 1970s as a Soviet grand project, abandoned half built, finished in the 2000s with a fortune from the American-Armenian collector Gerard Cafesjian. Botero's bronzes sit at the bottom and escalators run up inside past halls of glass art, Warhol and Chagall. From the top, on a clear evening, Ararat.

Take a driver and go east into the Azat gorge. Garni first: a colonnaded temple in the Ionic order on a spur above the canyon, built in the first century AD when Armenia was a client kingdom balanced between Rome and Parthia, kept as a royal summer residence after the country turned Christian, and thrown down by an earthquake in 1679. It was reassembled from its own fallen blocks in the 1970s, which is why the joints show, and it is the last classical colonnaded temple standing anywhere in the former Soviet Union.

Below the village, a rough track drops into the gorge to the Symphony of the Stones, a wall of hexagonal basalt columns hanging over the river like an organ, formed as lava cooled and cracked.

Then Geghard, ten kilometres up the valley where the road ends against a cliff. The front church dates from 1215; behind it, chambers, chapels and a mausoleum were cut directly into the mountain, with columns and ribbed vaults carved out of living rock rather than assembled from blocks. Sound does remarkable things in the upper chamber, and there is often a choir standing in it. The name means spear, for the lance kept here for centuries.

Give the morning to Tsitsernakaberd. Twelve leaning basalt slabs close over an eternal flame on a hill across the Hrazdan, beside a split stele forty four metres high. It went up in 1967, when acknowledging the deportations and massacres of 1915 was still an act of defiance inside the Soviet Union, and the museum cut into the hillside below is careful, documentary and very hard going. Two hours, and it is the right way round to have seen Baku first rather than after.

In the afternoon, the Matenadaran at the top of Mashtots Avenue, under a statue of the monk who invented the alphabet. It holds something like seventeen thousand Armenian manuscripts, much of it carried out of Anatolia by refugees a page at a time, and the display rooms show a hundred or so: illuminated gospels, medical treatises, one manuscript under twenty grams and another that takes two people to lift. Take the guided tour.

End above the Cascade at Mother Armenia, twenty two metres of woman with a sword on a pedestal more than twice as tall, raised in 1967 on the plinth where a statue of Stalin had stood until the thaw. Soviet artillery in the park, a Ferris wheel, and the best view in the city at sunset.

Armenia adopted Christianity as its state religion in 301, before anybody else, and this is where that happened. Echmiadzin, twenty kilometres west and conveniently on the airport road, is the seat of the Catholicos of All Armenians and is generally reckoned the oldest cathedral in the world, founded by Gregory the Illuminator on a spot where he said Christ had come down and pointed. The altar stands over what had been a Zoroastrian fire temple, and the treasury beside it holds the spear that gave Geghard its name.

St Hripsime, on the eastern edge of the town, is the more important building architecturally. Put up in 618 and named for a Christian woman martyred by the Armenian king before his conversion, it is the model Armenian church design copied for the next thousand years: a cross inside a rectangle, a dome on squinches, almost no ornament.

Then Zvartnots, minutes off the airport road: the ground floor of an enormous three-tiered round cathedral raised between 643 and 652, which collapsed within about three hundred years. A ring of column bases, capitals carved with eagles, and the outline of something far more ambitious than anything being built in Europe at the time. Three countries, twelve days, and this is the oldest thing you will have stood in.