Dermot O'Leary on The X Factor: “If I was Simon I'd probably take it off air for a year"

We meet in a north London gastropub, and he is literally bouncing with enthusiasm about his new show. His handshake is firm, his smile wide, his hair short, and at 42 he still looks like a young squaddy in Civvy Street.

O'Leary has got the knack of the golden soundbite. If TV ever failed him, there would always be a career for him in marketing. I ask him to describe The Getaway Car, which sees him move from ITV to BBC for a prestigious early Saturday-night slot. “At its heart, it's Total Wipeout with an element of Gogglebox chucked in.”

Like Total Wipeout, contestants take on a number of assault courses. But here they are driving – sometimes blindfolded. For added drama, they are guided by their loved ones. It's a recipe for chaos, smashed cars and domestics galore. In the final round, the surviving contestants get to take on the Stig for a £10,000 prize.

O'Leary admits they wrote off a handful of cars in the series, which was filmed in South Africa. It sounds like there's going to be a huge amount of tension in the show? “Yeeeeaaaaaah!” he roars. Are you out to destroy these poor families? He grins. “Only in a fun way. It's all very tongue in cheek. There's a real lovely warm feel to it.”

O'Leary's language is interesting. It's not the only time in the interview he talks of the importance of a show having warmth – something he has by the bucketful. I begin to sense he's making comparisons with The X Factor. Or, at least, with what The X Factor has become. “The Getaway Car doesn't take itself too seriously, and it's just a lot of fun. When you do a show, sometimes you kid yourself [about it being fun]. It's a siege mentality, it's all of us together, whereas here everyone just enjoyed themselves, no one moaned about long hours, everyone just came to work with a smile on their face, going 'it's quite funny this, isn’t it?' And that's what I loved about it.”

What he has loved is being involved right from the start. “It's the first show I've done since [Big Brother’s] Little Brother where you can set the tone of the show from day one. After three years of X Factor being made without me it took a while to put my stamp on it, whereas this.... it's a team, we all sing from the same hymn sheets, no one thought we were changing the world or lives. We were just having a good time.”

One reason he wanted to leave was to mix things up. Did he feel he was becoming stale? “A little bit. I lived for the live shows. And I was still having fun doing them, but it got to the point where I couldn't develop anything else. And you've only got one life, haven't you?”

There was a story that he jumped before being pushed after hearing they were looking for new presenters. Is that true? “No. I made the decision before that.”

In the end, he says it was an easy decision. Not least because some of his closest colleagues had left. Is he talking about the judges? He bursts out laughing. “Judges! Oh God, no! It's like being a civil servant: 'Oh right, it's a Labour government, is it? Right, OK, what have I got to do with that?' People I worked with very closely left, like series producer Mark Sidaway. Sid left and about a week later there was just something in the water and I thought now's my time.”

O'Leary appears to have chosen a good time to leave. Viewing figures were well down this past season – just over seven million watched the show's launch, compared with nine million in 2014. “If I'm honest, I don't know where those viewers have gone...” He pauses, and starts again. “Well I know where they've gone, they've gone and watched Strictly. But I don't know why they've gone.” It's a big admission from O'Leary. For so long Cowell et al have denied they have lost viewers to Strictly, arguing figures had fallen because viewers were watching on catch-up at their own convenience.

O'Leary says he hasn't seen much of this series because he has been filming The Getaway Car in South Africa. Then, without being prompted, he talks about the show's failings. “I watched the final. I thought the show was great, but I don't think they can say things like 'Change my life forever' and 'Amazing!' Just use different language.”

He's not complaining about the quality of contestants, though. “I think the talent's brilliant. Louisa's a great winner. Lauren could have been a great winner. I was cheering for Lauren.”

If you were Simon Cowell, what would you do now? “If I was Simon I'd probably take it off air for a year. I'd rest it, and then I'd get back to the rooms. The room auditions are great. For me it was always room, arena, boot camp. Also I was really uncomfortable about the Six-Chair Challenge. I don't think it's a particularly nice bit of television. One of the reasons people like Strictly is because Strictly is a very warm show, and I think you've got to keep that in mind.” Again that word – warm.

How does he get on with Cowell now? “I don't speak to Simon. But me and Simon barely talked when I was doing the show. We have a very professional relationship. We still do. We text from time to time. But I still care about the show and I want it to do well.”

A couple of years ago O'Leary said that sometimes he thought a presenter's talent was taken for granted: “You never want other people to fail, but sometimes you need a few .... cock-ups to remind people that it's a finely honed craft.”

Well, the presenters have certainly made a few mistakes in this year's X-Factor – not least when fellow Essex boy Olly Murs announced Monica Michael was going home before the official result had been announced. Did O'Leary experience any pleasure in that? “No!” he shouts. “No! Because they are my friends. All you do is feel for them. Don't get me wrong, it's nice to be missed but when it's your friends going through it, you just go, ‘Awwwww!’. I texted Olly straightaway...”

The Voice UK, The X Factor's rival, will be transferring from the BBC to ITV next year. Would he present it if asked? “No.” Why? “Because I've done that kind of show for eight years and I do want to do something different now.”

But O'Leary admits he can't get his head round the fact that The Voice has been criticised by some high up in the BBC who say that the Beeb's remit is not to provide popular and populist programmes. “I think it's good for the BBC to do a show like The Voice. It's a good thing that the BBC's popular.” O'Leary presents a popular Radio 2 show, and hears similar arguments. “I have the same thing at Radio 2, which gets it in the neck because a lot of people listen to it. You're sort of damned if you do, and damned if you don't. Radio 2 is the biggest radio station in Europe and that means it's serving a purpose. For me the greatest thing about the licence fee is that you've got 6 Music, you've got Radio 3, you've got Radio 2, the whole breadth of the BBC is important.” He sounds positively bewildered by the idea that The Voice was too successful for BBC. “The Voice was a successful show, which for me means it was doing something right. I don't think the BBC should get a kicking for that.”

As well as light entertainment, he says he'd love to do a show that makes the most of his interviewing skills. He has always been fascinated by politics. He once described himself as a socialist, but today he says that wasn't quite right. “I'm left of centre.” This year he presented a special election Question Time for youngsters, and he'd love to do more political stuff. “I love politics. I get really annoyed if I miss Question Time, or if I miss Prime Minister's Question Time.” The problem is, he feels pigeonholed after The X Factor. “When you've done one of the country’s biggest shows for eight years, there aren't many downsides, but that is one of them.”

Does he think it's possible to break that mould? “Yes. Look at Alexander Armstrong; someone who can do Pointless and do a documentary and then do Have I Got News For You. He makes great career choices and does what he wants.”

Where would he like to be in five years’ time? “If I'm still working and I've got a good body of work, I'll be happy.” And outside work? Well, yes, he and his wife, TV producer Dee Koppang, would like children, but he prefers to keep his private life private.

Since leaving The X Factor, we've seen little of him on telly. Has that been a deliberate decision? Absolutely not, he says. “Because I quit a bit earlier than I thought I would, I started from a standing start. I haven't done less, I've just developed more. I’ve actually been busier than I have been since I was 24. It's been like being a kid again. It's fantastic.”