merrybee91's Book Journey

Wtf did I just read? Is this nonfiction or sci-fi? I’m only half joking. The writing tone throughout this book often reads like a chilling horror sci-fi novel. There’s a constant back-and-forth between “fable” and “speculation” and “oh by the way this next part has already happened in real life history,” and to be honest, it gives whiplash.

And actually, I didn’t realize/remember when I picked up this book, but I had heard of Eliezer Yudkowsky before, very briefly mentioned in Karen Hao’s Empire of AI (which I absolutely recommend miles above and away over Yudkowsky’s book). Turns out that Eliezer Y. is part of a philosophical school of thought known as “effective altruism,” which I would never have known anything about from reading his book, because the term doesn’t even make an appearance. (Hmm…)

In Empire of AI, Hao states that

[…] the movement […][of effective altruism] preaches dedicating oneself to doing maximal good in the world by using extreme rationality and counterintuitive logic to guide decisions[.]

Then Hao goes on to say:

But this existential brand of AI safety, built on philosophical thought experiments, would soon come under fire as the AI research community awakened to the less apocalyptic and immediate real-world harms of AI.

And therein lies the crux of my major issues with Yudkowsky’s takes in If Anyone Builds It, Everyone Dies.

In short: Yudkowsky hates AI because of his Doomer computer calculations about sci-fi-esque possibilities regarding hypothetical super-intelligences in some unknown future who will have “weird, strange alien preferences.” I hate AI because of all the real-life human damage it’s already wreaking right now and the real-life humans who are profiting carelessly. We both hate AI, but we are not the same.

That’s not to say that this dude doesn’t make some interesting, well-founded points. By all means. And if this is the book that gets you to be anti-AI... great, I guess. It's a start. But Yudkowsky' argument tends to rely heavily on his ability to repeat something in a chilling, condescending, mansplainy tone over and over and over, rather than basing his words in actual fact or possibility.

Even as he puts this argument together about how Artificial Super Intelligence will work in a way that’s so fundamentally different from human intelligence, he’s projecting so much freaking male humanity into his predictions! This book showcases a mind that is completely lost in a theoretical world of rationality, so self-assured of its own understanding of everything about how the world works, that it has created a prison where it's now pacing back and forth restlessly and screaming at us all about AI doom because this mind is far too shuttered and corralled to be able to see any other options that might be staring it in the face.

This man is so blindly convinced of full human supremacy currently that it’s astounding:

Humans are no longer the world champions at chess. Humans are no longer the planet’s only language-users.

The presuppositions in these statements are wild. I studied Linguistics so I get it, humans just looove to think that we are the only living creatures with language, and that makes us feel special, it makes us feel separate from all the other living species who only have “communication systems,” not languages. Never mind that our definition of what makes the human communication system a “language” is mostly a lot of arbitrary categorization and systematization that seems to continuously need further refinement to distinguish why it’s not an animal communication system with every new discovery science makes about the complexity of animal and natural communication systems. But okay, whatever.

Yudkowsky flippantly passes by other intelligent minds and consciousnesses, such as octopus and trees, without any real acknowledgement of how absolutely crazily fascinating those two particular consciousness / communication systems actually are.

Re octopus:

Humans aren’t necessarily the best at everything; maybe an octopus’s brain is better at controlling eight arms. But in some broader sense, it seems obvious that humans are more general thinkers than octopuses. We have wider domains in which we can predict and steer successfully.

Yes, dude, it seems obvious to you because you clearly know nothing about an octopus and you clearly don’t spend much time thinking about things that fall outside your own realm of understanding the world. Also, “it seems obvious” is literally not a real argument? That wording is the opposite of fact? Hello?

Then there’s this bit about trees:

Trees are made mostly out of air. They use sunlight to strip carbon atoms from CO2 molecules and arrange those atoms into bark and branch. Physics permits the possibility of technology that uses sunlight to spin air into wood. Could a superintelligence invent that technology? Almost surely. Trees are produced by running RNA strands through ribosomes to produce proteins (in a suitable cellular environment). Human labs can use ribosomes too. So the challenge of building custom-designed biological technology is not so much one of producing the tools to make it, as it is one of understanding the design language, the DNA and RNA.

So what you’re telling me is that trees are literally already doing this thing that you’re extremely terrified that ASI will do, and rather than killing us, they’re giving us actual life. Let’s not even mention that meanwhile, humanity is actively killing trees for… checks notes… oh yes, expansion of this AI technology that might want to kill us all. IDK, seems like the real monsters here might just be the humans. Maybe we should let the trees take over at this point, since their technology seems to be more advanced than ours, so they might have a better shot at taking care of the AI issue than we do. Not to mention the insanely incredible amazing mind-blowing communication system (not language, of course) that trees use to connect with each other and send messages across entire forests. Yep, my bet’s with the trees at this point. (And mushrooms, but Yudkowsky doesn’t mention anything about the mycelium, so I guess that particular non-human intelligence is beyond the scope of his book/understanding/imagination.)

For me, the real horror story of AI lies in between the lines of what goes unsaid in Yudkowsky’s book. For example:

It [ASI] still needs humanity to keep the supply chain running—the supply chain that ultimately mines metal and forges them into robots and computer chips.

BRO. This is not hypothetical. What do you think has been the source of so much war, violence, exploitation, and atrocity across countless places in the world that have been ravaged by colonialism and extraction?? At one point, the author postulates that ASI would “let the world get hot.” Bro, again. Not hypothetical. Real world human leaders and tech bro billionaires are letting the world get hot as we speak (so they can build AI). At another point he skims breezily past the labor of training AI’s which I learned from Karen Hao’s Empire of AI is an entire real-life horror story of its own.

I was glad that I pushed my way through to the end of the book only because I was rewarded with this beautiful C.S. Lewis quote, that I will leave for you here so that you need not subject yourself to the same experience just to get this lovely quote:

“How are we to live in an atomic age?” I am tempted to reply: “Why, as you would have lived in the sixteenth century when the plague visited London almost every year, or as you would have lived in a Viking age when raiders from Scandinavia might land and cut your throat any night; or indeed, as you are already living in an age of cancer, an age of syphilis, an age of paralysis, an age of air raids, an age of railway accidents, an age of motor accidents.” If we are all going to be destroyed by an atomic bomb, let that bomb when it comes find us doing sensible and human things—praying, working, teaching, reading, listening to music, bathing the children, playing tennis, chatting to our friends over a pint and a game of darts—not huddled together like frightened sheep and thinking about bombs.

TLDR: Skip this book, and read Karen Hao’s Empire of AI and/or Shannon Vallor’s The AI Mirror instead.