Preheat the oven to 170°C/325°F/gas 3. For the custard base, bring the milk and all the cream just to the boil in a saucepan. Cut the vanilla pod in half, scrape out the seeds and add to the pan. Whisk the eggs with the sugar until pale, then whisk in the milk and cream mixture, removing the vanilla pod shell.
Dip the hot cross bun halves in the mixture, then place in an ovenproof dish. Drizzle over the Cognac and sprinkle over the apricots and the orange zest. Sieve the custard over, and leave it all to soak for at least 15 minutes.
Place the dish in a roasting tin, half-fill the tin with hot water, then bake the pudding for about 45 minutes. When cooked it will have a slight crust on top but will still be slightly wobbly inside. Dust with the icing sugar and serve.