J. E. Landau

Yesterday, the AI company Anthropic announced something called Project Glasswing, a collective effort to patch security vulnerabilities that the company’s AI models have supposedly discovered “in every major operating system and every major web browser”. One of my friends, jokingly, talked about hackers getting a bootleg copy of this system and using it to hack like they do in video games or movies.

And this kind of automated exploit development system would be a world-changing technology if it exists. Imagine hackers being able to find zero-day exploits and attack systems without needing to do any of the grunt work of investigating themselves. A single person could hack into random things, and they’d be practically unstoppable. Because any human defenders would be left behind.

Of course, it probably wouldn’t end up working like that. The existence of Claude Mythos isn’t a secret, and Glasswing hopes to patch many of the security issues that Mythos is discovering as they are discovered. I am a bit skeptical of Anthropic’s claims, because the AI industry likes to make very big claims, and I don’t think that all of these claims are substantiated, but time will tell either way.

Regardless, I think that this incident reflects a broader issue in Science Fiction. People like to write stories where just one person has a cool toy, and nobody else has anything like it.

One example of this is in Neal Stephenson’s novel The Diamond Age, where the engineer Percival Hackworth creates illegal copies of a device called a “primer” for his daughter, Fiona. Hackworth is the lead of the project to create the device, which uses highly advanced nanotechnology to educate a young child, but which also has an extremely sophisticated computer system that can do things like hacking and contracting actors to help create educational footage. This is accomplished with stores of digital currency, which is not that unrealistic, but the device proves to be an extremely powerful educational tool.

Additionally, copies of the primer are made later in the story, and distributed to millions of children in China, in an attempt to create a powerful and highly-educated generation. But, this project is hijacked when one of the children with an original copy of the primer, which is able to control the educational content in the other primers and ultimately hijack major pieces of nanotechnological infrastructure, making its user one of the most powerful people in the world.

This is, of course, only possible because it was a system that was created in secret and only distributed to a limited number of people, at least at first, and somehow these original versions are able to stay ahead of and control later versions. The Diamond Age is an excellent novel and I am quite fond of it, but I don’t think that this reflects the course of real-life technological developments.

Another book with a surprising, highly-advanced technology is The Moon is a Harsh Mistress by Robert Heinlein. There, a supercomputer called Mike spontaneously achieves consciousness once it becomes large enough, and ends up being a key part in a bloodless revolution that frees the moon from Earthly control. But no clear reason is given why Mike is the only such intelligence, or why Earth doesn’t have something like Mike, or why nobody expects the existence of Mike. A lot of the things that Mike is able to achieve rely on him being unique and unexpected, but I don’t think that the novel does a good job of explaining why this is, and I think that it doesn’t reflect the nature of real-life technological developments.

For a third example of this sort of super-powered technology being depicted in an unrealistic way, I think of Margaret Ball’s Disappearing Act, which is a somewhat obscure science fiction novel that I had the pleasure of reading a few months ago. In Disappearing Act, many neurological problems can be solved using a living tissue graft called a “bacteriomat”, which can only be made on a specific planet due to growing in specific physical conditions. This is sensible to me, but the crux of the plot is that a criminal organization has found a way to grow more bacteriomats, by using the brains of living humans in a torturous process that ultimately kills the victim. Since bacteriomats are so valuable, and the demand is depicted as being very high, it seems unrealistic to me that only this single criminal group would figure that out, especially given the large number of ruthless organizations depicted in the story.

Having an unrealistic depiction of technological development doesn’t make a work bad. I enjoyed all three of these novels, and The Moon is a Harsh Mistress is one of the defining novels of its era of science fiction, although I don’t necessarily endorse its political content. Still, I think that we can do better, if we look at historical developments of advanced technology.

An example of this is the atomic bomb, which was developed in secret in the United States. First, the US wasn’t the only country to consider the possibility of the atomic bomb. Nazi Germany had its own nuclear bomb project, and pursued it at the same time as the US and Japan had a small pilot program, while the Soviets would be forced to play catch-up. The US nuclear monopoly would only last from 1945 to 1949, when the Soviets would test their own nuclear device. The US would then attempt to prevent proliferation in most cases, but nuclear weapons would spread around the world, and to date at least ten nations have developed nuclear weapons—including the relatively poor and isolated South Africa. More nations had what is called “nuclear breakout capacity”, where they did not possess working nuclear weapons, but made a point of keeping around most of the parts and know-how to build them in a hurry.

In this vein, a more realistic case of technological advantages in science fiction is William Gibson’s Neuromancer and its sequels, which has an artificial intelligence orchestrating a daring heist to allow it to absorb another intelligence, the eponymous neuromancer, and become a godlike superintelligence. This superintelligence is able to operate in secret for a while, it exists without competition for some time, and it proves to have world-changing effects. But this was not unexpected, and the heist is only necessary because there exists a government organization specifically dedicated to preventing the emergence of godlike AI.

Eventually, competitors are developed, the technologies become known to more people, and then the world reaches a new equilibrium and balance of power, as with nuclear weapons in real life. In the sequels to Neuromancer, the superintelligent AI fragments into pieces called Loa, with varying goals, and human hackers are able to use heavy cybernetic and physical modification to become new superintelligences, showing some of the same capabilities as the original fused AI.

Other times, new technologies become possible, but their application is restricted to certain areas because of specific local factors. One example of this, historically, was the development and proliferation of agriculture in Europe and the Middle East which required the confluence of rivers. In places that did not have this geographic, hunter-gatherers dominated until the proliferation of technologies that allowed for the farming of more hostile environments.

A book that has this sort of resource-restricted technological advantage is Anti-Ice by Stephen Baxter, where the United Kingdom gets access to a metastable form of antimatter during the Victorian era, and is able to harvest it and retains a monopoly on this material, partly with the use of antimatter bombs and partly due to its remote location in Antarctica. Still, even this monopoly is imperfect, and in the course of the story foreign powers are able to gradually secure a small supply of antimatter. Still, while it lasts, the United Kingdom is virtually unstoppable.

Finally, I think that a third case for plausible technological advantage in science fiction is when it is artificially enforced through social means. Historically, many advanced technologies have been delayed or prevented by social pressure, such as nuclear power. Civilian nuclear power is often an early stage in a military nuclear program, and there also exists significant fear around nuclear power due to a few high profile disasters, even though nuclear power is overall quite safe, with a death rate only second to solar per unit of electrical energy produced. Nuclear power is still used, but it is tightly regulated, and nowhere near as prominent as it could be, if the world had more positive views on the technology, and some countries have more nuclear power than others, probably due to differences in social attitudes towards the technology.

An example of this in science fiction might be Julian Comstock: A Tale of 22^nd Century America, where the United States has an intentionally technologically regressive society that suppresses innovation to return to an idealized (and dystopian) vision of 19^th century Americana, with steam technology being the most advanced available and other countries, such as those of Europe and China, having much more advanced technology. But, having more advanced technologies would disrupt the social fabric, which relies heavily on feudal relationships and indentured servitude.

More broadly, I think that it’s pretty common for science fiction stories to center on the development of new technologies. It’s the origin of the genre, going back to Jules Verne and H. G. Wells, and it definitely makes for compelling stories. But I think that there is a desire as a science fiction writer to have your cool new technology change a world that barely reacts to it, which I don’t believe is justified in most cases.

In the 21^st century, governments are constantly looking out for disruptive technologies and private theorists (including science fiction writers) are constantly making up new ones, realistic or not. There is an enormous race to find what the next big world-shaking thing will be, and because of this I think it’s highly implausible that you could end up in a situation like Daemon, where a single genius’s invention conquers the whole world and destroys all pre-existing systems.

Instead, I think that if we are going to write new-technology stories, we should carefully consider how the world will react to those technologies, usurp them, and try to mitigate their effects, instead of passively allowing their ways of life to be destroyed.