Nate Clark's Musings

Recent advances in AI have been pretty breathtaking. Understandably many pundits are concerned about a rapid take-off, where the AI learns to self-improve and gets exponentially more intelligent to the point that it could destroy humanity.

I’ve long been skeptical of this possibility for the fact that computation done in silicon is wildly less energy efficient than what happens in the human brain (fwiw, my Ph.D. dissertation was on how to make silicon computation more efficient). Quantum computing, adiabatic circuits, or some other exotic computer architecture might change all this, but let’s do some digging and see if silicon intelligence in modern architectures could in fact theoretically scale up enough to be significantly smarter than humans.

Refining that question to something we can answer is actually really difficult. There’s no consensus on how brains work or how much computation they do (a lot of people hate the idea of equating brain function to computation altogether). There’s also no great definition of intelligence, no models for determining ex ante how “smart” an AI will be given an input architecture, or even any consensus on what we’d have to model to replicate brain function. Sam Altman, CEO of OpenAI, is on record stating that current AI architectures won’t really scale up beyond their current size, but no one knows what the best models look like.

For the sake of argument let’s assume the brain is a computer and we could construct a replica in silicon. Estimates I’ve seen for how many computations a brain performs range from about 100 teraflops up to 10 petaflops. Silicon is unlikely to get a whole lot more efficient, and current GPUs can to roughly 300 teraflops at 400 watts, so on its face this looks scarily close to being comparable to human brains. Yikes!

But it turns out that computation is probably not the limitation we should look at. Moving data around to the different computation units very likely takes more energy than the actual computation. This makes intuitive sense, since the cost of adding one unit of computation will quickly be dwarfed by the cost of adding communication paths from the new unit to all the existing ones. Studies have shown that 35X more energy in the brain goes to signal transmission between neurons rather than to the actual computation they perform. Similarly, silicon chips also tend to be dominated by interconnect, i.e., the wires that move bits around between logic gates.

Researchers have looked at using different benchmarks that put more emphasis on shuttling data between computation units and found that the human brain is roughly 1-30 times more powerful than the fastest supercomputer. I find that estimate pretty compelling, and even if it’s off by an order of magnitude, that’s very buildable.

So I was completely wrong. It is theoretically achievable for AI to achieve parity with humans with current silicon computing techniques. Or at least not obviously impossible.

The one saving grace is that current AI systems don’t actually replicate computation/interconnect in the brain. They do something approximating what we think happens in there but probably much less efficiently. There are reasons to believe that brains are within an order-of-magnitude of the theoretical limit for compute efficiency, where current AI models are renowned for requiring significantly more compute/memory to run. There’s probably an order-of-magnitude or two of efficiency difference due simply to the architectures AI engineers use. That will help me sleep at night. But I certainly wouldn’t bet against clever people closing that gap in my lifetime and bringing about the robot apocalypse.

Well, if this is still on the internet when that happens, I’d like to be the first to say…