Why AI needs muggles

21 September 2026

Aidan Rave argues that the future of artificial intelligence should not be determined solely by people who understand the technology best

 

Points raised:

- AI warnings have intensified, often from those who accelerated progress 

- Expertise isn't governance; specialists shouldn't govern themselves alone 

- Non-technical "muggles" ask vital first-principle questions 

- AI's direction needs diverse perspectives, not narrower technical control

 

I took the time to read Dario Amodei’s fascinating essay over the weekend, the latest in a series of high-profile interventions on the progress of AI, which have included a genuinely diverse range of contributions from tech bros to the Pope.

In recent weeks the urgency and tone of these comments has intensified noticeably, leading to a steady stream of warnings from the people leading the race to build ever more powerful artificial intelligence and those in wider positions of influence.

Some of those warnings may prove prescient. Some may ultimately prove overblown. Either way, they raise important questions.

The irony, however, is difficult to ignore. Many of the loudest calls for caution are coming from the same people who spent the last decade accelerating progress. It's a little bit like the organisers of a Formula One complaining that it’s dangerous because the cars are going too fast! 

The debate often centres on what AI will become. How intelligent it might be – will it cure cancer in the new few years? How much of our work it might automate and therefore what the consequences might be for people’s livelihoods. More significantly, we’ve heard sinister warnings about the percentage chances of AI extinguishing humanity all together. Suddenly Terminator 2, 2001 a Space Odessey, Westworld and the like feel less like light entertainment and more like a chilling vision of our near future. 

All of these are clearly important questions, but I’m not convinced they are the most significant questions. 

Can we? Should we?

Governance has always concerned itself with a different question. Not just what is possible, but what contributes to purpose, value and outcome, which in the case of AI, is presumably the progress of humanity, rather than our imminent demise. 

Anyone who grew up in the 1990s will remember the immortal words of Ian Malcolm, played by Jeff Goldblum, in Jurassic Park:

"Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should."

It remains one of the greatest governance observations ever committed to film.

The problem wasn't that the Jurassic Park scientists lacked intelligence. Quite the opposite. They were brilliant. The problem was that everyone around the table shared a similar frame of reference. They knew how to create dinosaurs. Nobody seemed particularly responsible for asking whether resurrecting apex predators was an especially sensible idea. We see a version of this playing out to this day in boardrooms up and down the land, from the mundane to the occasionally catastrophic: “Of course the software programme is working properly, the problem is just a few dishonest postmasters”.

The challenge facing artificial intelligence today feels remarkably similar.

Expertise is not governance

The AI sector is full of exceptionally talented people. Researchers, engineers, mathematicians and entrepreneurs all pushing the boundaries of what technology can achieve. Society already benefits enormously from that expertise. However, expertise alone is not governance.

If history teaches us anything, it is that specialists should never be left entirely to govern themselves. After the financial shock of 2008, does anyone seriously think it’s a good idea for banks to be governed solely by bankers? Similarly, should hospitals be governed solely by clinicians? Should the police, police themselves? Quis custodiet ipsos custodes as we often say around Doncaster way. 

Of course not.

The purpose of objective oversight is clearly not that outsiders know more than experts, it is that they know different things.

Good governance emerges from the interaction between specialist expertise and broader human judgement. Which brings us to what I will unapologetically call the Muggle problem.

The AI debate is increasingly conducted by people who understand the technology at a depth that most of us never will. That creates a risk. This risk is escalated at a point where the AI actually knows more than the architects themselves – a point which suddenly and perilously, feels much closer than it once did. 

It’s not that these individuals are necessarily untrustworthy, nor that they are purely acting in bad faith. I’m sure most are acting with the best of intentions. The much bigger issue is that every specialist community, however clever and well-meaning, will inevitably develop assumptions, blind spots and forms of groupthink.

The more specialised the discipline becomes, the more important it is to expose decisions to people who think differently.

Muggles matter

That’s why the muggles matter. Not because they understand transformer architectures or because they can explain large language models, but because their relative ignorance enables them to ask different questions – first-principle questions, occasionally derided as ‘daft questions’ (which of course are never daft). These questions might be:

  • What problem are we actually trying to solve?
  • Who benefits most?
  • Who bears the risks?
  • How will this affect ordinary people?
  • What happens if we get this wrong?
  • Who is accountable?

Such questions rarely emerge from technical expertise alone. They emerge from effective oversight contributed in a psychologically safe and specifically designed environment rooted in the core principles of good governance.

When AI leaders argue that the technology is becoming too important to ignore, they are almost certainly right, but the answer cannot simply be to hand greater influence to a smaller and smaller group of technical specialists.

If anything, the opposite is true.

The more important AI becomes, the more important it is that non-technical voices have meaningful influence over its direction. In The Lord of the Rings, it is not the most powerful wizard who ultimately carries the burden of shaping events. The fate of Middle earth ends up resting with a collection of unlikely companions who can see the world from very different perspectives.

Different problem. Same principle.

Diversity of perspective is seldom a weakness in decision-making; indeed, it is often the thing that can prevent catastrophic mistakes.

Here’s the point. The future of artificial intelligence should not be determined solely by people who understand the technology best. It should also be shaped by people who understand humanity. Because whilst artificial intelligence may continue getting smarter, governance remains the mechanism by which societies decide what that intelligence should actually be used for.

One final film reference. In Days of Thunder, if AI innovation is the engine, then governance must ultimately be the steering wheel and, when necessary, the brakes. 

At this crucial moment, we need more people holding the wheel, more people with their foot across the brake, not fewer.

Meet the author: Aidan Rave

Principal consultant and partner

Email: aidan.rave@good-governance.org.uk Find out more

Prepared by GGI Development and Research LLP for the Good Governance Institute.

Enquire about this article

Enquire
Contact us