The Coming Wave, by Mustafa Suleyman

Ernest Oppetit

Dec 29

This book was so heavily promoted across the internet I had to read it. Just like how I succumbed to an Athletic Greens subscription after the deluge of podcast ads.

Mustafa "Moose" Suleyman is particularly clued-in on AI and the regulation landscape. He's a co-founder of Deepmind and someone who is very plugged in to the regulatory circles (he is one the 7 AI leaders Biden summoned for a chat in July). I didn't know he also had a background in mediation and social enterpreneurship before Deepmind.

Overall I enjoyed it. It was a good high-level overview of the wave of technologies coming online -- AI, synthetic biology, quantum -- and their promise and perils, their centralising and decentralising force. The nation state is both being bolstered (i.e having greater surveillence capabilities) and challenged by these technologies pushing power "down" towards people and groups (such as Hezbollah). This didn't start with this wave, the internet has already started this as Martin Gurri explains in The Revenge of the Public, but it definitely feels like it's accelerating.

There were some interesting policy recommendations such as enforcing auditability and transparency for larger systems, and pushing states & critics to build AI systems themselves to improve the quality of the debate. Moose argues we must walk the "narrow" and messy path of letting these technologies thrive while trying to mitigate the worst downsides, which will take involvement from all parts of society.

Onto the notes, enjoy!

Part I: homo technologicus

Familiar whirlwind tour of the story of technological waves, better covered in Homo Deus
Unintended consequences. Gutenberg wanted to print more Bibles but caused the Reformation. Fridge makers didn't want to make a hole in the ozone layer with CFCs. Internal combustion engine makers didn't want to cause global warming - they argued for an environmental benefit in the early days: less horse dung spreading around the city.
"Have we ever said no?": printing press, knitting machines, all modern weapons, all tech eventually gets used despite initial uproar
"Technologies are ideas, and ideas cannot be eliminated"

Part II: the next wave

Chronicling the recent progress in AI
Running through the recent progress in synthetic bio, CRISPR, nanotech
- Covered in my review of The Code Breaker
Adding into the wave: robotics/drones, quantum, new energy sources (notably Fusion)

Unlike the waves of the past, this wave has four new features which makes containment harder. All accelerating due to incentives: pure commercial opportunity, arms races, ego

Asymmetric impact: the tech is widely available, a single person in their garage can start a new pathogen, a new AI algo, or use drones to combat.
- Or a new cyberattack, for example WannaCry which caused $8B of damages (covered in my review of Sandworm)
- Tokyo subway Sarin attack. 13 dead, 50 severely injured, 1000 injured
Hyper-evolution: the fields reinforce themselves, AI used for synthetic bio, for robotics, energy costs decreasing reinforces them etc.
Omni-use: even broader than "dual-use", can be used for research, warfare, civilian life. General purpose tech
Autonomy: human in the loop, for now
- Iranian Nuclear scientist Fakhrizadeh was possibly killed by an automated machine-gun in 2020. There is a debate about whether it was fully remotely operated or not.

Part III: states of failure

The nation state is undergoing a crisis. Democracies are built on trust and trust is collapsing.
- 2018 study: one in five Americans said "army rule" is a good idea
- Between 1980 and 2021, the share of income earned by top 1% has doubled to 50%.
- 40M people live in poverty in the US.
Harder to contain/secure society. Power is dipersing. Drones, information warfare / deepfakes, lab leaks, cyber.
Temptation of surveillance state. China obviously leading the way.
"States within states"
- What is Hezbollah? Political party, but also has an army, operates schools, hospitals, etc. inside the Lebanese state.
- The Coming Wave makes it easier for small state-like actors to thrive. More fragmentation.
Paradoxical push towards both centralisation and decentralisation. Easier for the state to monitor, surveil. Easier for new groups to form and become powerful: "Hezbollahisation"
Stagnation not an option due to population growth and resource constraints

Part IV: through the wave

Regulating is very challenging. Motorised transport has regulations around traffic, roads, parking, seatbelts, emissions, driver training etc. from several bodies. 1.35M people still die every year in traffic accidents

"Ten steps towards containment"

An Apollo program for technical safety. Figure out sandboxing / off switch. "Provably beneficial AI". Less than 2% of researchers are focused on safety today
Audits. External scrutiny / red teaming. "Some level of surveillance will be necesssary"
- Learn from SecureDNA "a free, non-profit screening platform designed to safeguard DNA synthesis everywhere."
Choke points to buy time. Today there are some like TSMC and ASML (covered in my review of Chip War)
Critics should be practitioners.
Businesses to take responsibility (profit & purpose)
Governments: survive, reform, regulate. Feynman "what I cannot create, I do not understand". Governments should build AI systems
Alliances / treaties.
Culture: respectfully embracing failure.
- Aviation industry took a long time but as of 2010 1 death per 7.4M passenger boardings. In large part thanks to the culture of the industry
Movements: public input at every level
"Coherence" ensure all these points are working together (didn't quite get it)

Ultimately we must find a way to walk the "narrow path" which enables us to leverage these technologies while containing the worst downsides.

Finally, regarding the inspiration for the name of the book, I was not aware of this incredible story. From John Thornill's review in the FT:

When the South Korean political activist Kim Dae-jung was jailed for two years in the early 1980s, he powered his way through some 600 books in his prison cell, such was his thirst for knowledge. One book that left a lasting impression was The Third Wave by the renowned futurist Alvin Toffler, who argued that an imminent information revolution was about to transform the world as profoundly as the preceding agricultural and industrial revolutions. "Yes, this is it!" Kim reportedly exclaimed. When later elected president, Kim referred to the book many times in his drive to turn South Korea into a technological powerhouse.