#251 – The UK's former head AI safety scientist on how to solve alignment before superintelligence arrives | Geoffrey Irving

<p>When should governments slow the race toward superintelligence? According to <a href="https://naml.us/">Geoffrey Irving</a>, the careful answer is sometime in the past. The useful answer is now.</p><p>Geoffrey — formerly a safety researcher at OpenAI and Google DeepMind and chief scientist at the UK <a href="https://www.aisi.gov.uk/">AI Security Institute</a> — expects full-blown superintelligence in roughly two to three years.</p><p>***<br><em>Want to work with Geoffrey to help align superintelligence? </em><a href="https://jobs.80000hours.org/?refinementList%5Bcompany_data%5D%5B0%5D=Resolution&amp;utm_source=80k_podcast&amp;jb_source=homepage"><strong><em>Resolution is hiring!</em></strong></a> <a href="https://80k.info/work-at-resolution"><em>https://80k.info/work-at-resolution</em></a><em><br>***</em></p><p>The leading AI companies all have broadly similar plans for keeping superintelligence under control:</p><ul><li>Train models to have good character</li><li>Use increasingly capable AIs to supervise other AIs</li><li>Monitor them closely for signs of deception or scheming</li></ul><p>Geoffrey thinks that combination <em>could</em> work. The alarming part is that nobody has a strong argument that it <em>will</em>. He expects a crucial “phase shift” as models move beyond human intelligence:</p><ul><li>Below that threshold, humans can usually tell whether a model’s work is good and correct its mistakes.</li><li>Above it, the models themselves will increasingly determine the feedback used to train their successors.</li></ul><p>In this episode, Geoffrey and new host Tom Reed explore what might go wrong with the companies’ plans; why Geoffrey’s new nonprofit, <a href="https://resolution.org/launch">Resolution</a>, is pursuing a portfolio of neglected research bets; and whether governments should slow AI development while we work out which methods can actually be trusted.</p><p><em>This episode was recorded on June 29, 2026.</em><strong><em></em></strong></p><p><a href="https://80k.info/gi"><strong>Full transcript, video, and links to learn more: </strong>https://80k.info/gi</a></p><p>Chapters:</p><ul><li>Cold open (00:00:00)</li><li>Meet Tom Reed — our newest host! (00:00:32)</li><li>Who’s Geoffrey Irving? (00:00:59)</li><li>What misaligned superintelligence will look like (00:01:38)</li><li>Why are AI companies more optimistic about alignment than Geoffrey? (00:12:30)</li><li>Why Geoffrey expects superintelligence in 2–3 years (00:28:05)</li><li>When and how to slow down frontier AI development (00:31:30)</li><li>Safety researchers can have more impact in governments than companies (00:39:22)</li><li>How Geoffrey’s new organisation plans to tackle alignment (00:46:55)</li><li>Post-ASI science: nanotech, solving ageing, and uploaded minds (00:50:29)</li><li>Why we should expect superintelligence to accelerate scientific progress (01:03:30)</li><li>Can good character training carry over to superintelligence? (01:11:03)</li><li>What the field of AI alignment still doesn’t know (01:16:44)</li><li>Lessons from politics on how to combat power seeking (01:24:36)</li><li>Solving Pentago and working at Pixar (01:29:22)</li><li>Geoffrey’s best prediction (01:32:40)</li><li>Geoffrey’s best bets on which alignment techniques will work (01:37:38)</li><li>Work with Geoffrey at Resolution (01:43:34)</li><li>The dangerous asymmetry between capabilities and alignment (01:54:17)</li></ul><p><br></p><p><em>Our production team includes: </em></p><ul><li><em>Video editors: Josh Alward, Dominic Armstrong, Jasper Luithlen, Milo McGuire, Luke Monsour, and Simon Monsour</em></li><li><em>Producers: Elizabeth Cox and Nick Stockton</em></li><li><em>Coordination and support: Katy Moore and Lou Moran</em></li><li><em>Camera operator: Jeremy Chevillotte</em></li></ul><p><em>Music: </em><a href="https://open.spotify.com/artist/4lWobp6IUcSZ7w5mhnU1c9"><em>CORBIT</em></a></p>