Preheat the oven to 180°C/350F/gas 4. Lightly butter the bread on one side, then cut each piece in half diagonally.
Whisk the sugar, vanilla seeds and eggs together in a large bowl until well combined, then pour in the cream and milk and continue whisking until smooth. Peel and roughly slice the bananas, then roughly bash up the pecans and chocolate, if using.
Rub the inside of a baking dish (roughly 25cm x 30cm) with a little butter, then layer up the bread (butter-side up), bananas, pecans and chocolate, if using, finishing with a final layer of bread (again butter-side up). Drizzle over the Baileys and custardy mixture, then leave to stand for around 20 minutes, or until the bread begins to soak up the liquid.
Bake in the hot oven for around 35 minutes, or until set around the edges but still wobbly in the middle. Remove from the oven and allow to cool slightly, then serve as is, or sprinkle over the caster sugar and blast with a blowtorch to caramelise the top. Delicious served with good-quality vanilla ice cream.