Danger of Technological Empowerment

I have written a few times about why AI makes me nervous, but I think AI is actually one instance of a more general problem: technology keeps giving individuals more capability, and reducing how many people they need to cooperate with. This is basically the point of technology.

A person with a laptop can publish something, write software that controls machines, make fairly sophisticated parts, or access almost any human knowledge. This is great. As a technology libertarian, my default position is that people should be able to own powerful computers and experiment, encrypt, modify, run anything on anything they own, without seeking permission.

I think this logic has an endpoint that I don’t like.

The cooperation requirement

I talked about part of this in [[Mentor_in_the_Machine]]. Historically, doing something extremely bad has required unusual competence or a bunch of other people.

If I want to build a modern house, I need contractors, suppliers, inspectors, equipment, materials, and money. If I want to build a train, the list gets longer. If I want to build a nuclear weapon, I need a government sized organization.

All those people are friction, which is normally a downside for trying to do a thing. Somebody can refuse, ask why I need something, or call the police. Somebody may just be bad at keeping secrets. The project has to get approval from other people.

A lot of technological progress consists of removing exactly this requirement.

Computers let one person do what used to require clerks, researchers, draftsmen, accountants and programmers. CNC machines reduce the number of people needed to manufacture complicated parts. The internet removes a lot of the human chain involved in acquiring knowledge. AI will remove much of the need for expert help.

For almost everything people want to do, that is great. But some people are idiots and want to do things like make anthrax.

The megaidiot problem

The problem isn’t really that there are large numbers of evil people. But there are +eight billion people.

It doesn’t matter that 99.999% of people would never intentionally do something catastrophic with a technology. We still have the 0.001% x 8 billion.

Somebody will be angry / suicidal / in a cult. / an ‘invincible’ teenager. (I made worms, scraped credit card numbers off networks, and built a phreak-box as a curios kid… but all that damage was contained.)

Somebody will just be an megaidiot.

Society currently survives stupidity largely because one average person can only break so much. Guns are dangerous, but one person cannot kill Philadelphia. Cars are dangerous, but crashing it does not destroy Pennsylvania. At some point of personal empowerment, that protection goes away.

AI just happens to be the first general purpose technology (GPT) that I think is likely to do this.

But the underlying problem isn’t AI… it is increasing individual capability.

Defense doesn’t have to lose by much

The easy answer is that technology also makes defense better. AI can find software vulnerabilities and also patch them. Sensors get cheaper. Detection gets better. Drones can defend as well as attack.

But I think offense scales faster than and defense. For a lot of scenarios the defense has to close every hole, while offense only needs to find one. A hack or terrorist attack that fails 99 times and kills people on the 100th still doesn’t change the concern above.

This success rate is only manageable while individual ability is limited.

Everything is dual use

The distinction between civilian technology and weapons is fuzzy. There is an early scene in Harry Potter and the Methods of Rationality which emphasizes this to the extreme:

“Mr. Potter, all things have their accustomed uses. Give me ten unaccustomed uses of objects in this room for combat!”

“There are desks which are heavy enough to be fatal if dropped from a great height. There are chairs with metal legs that could impale someone if driven hard enough. The air in this classroom would be deadly by its absence, since people die in vacuum, and it can serve as a carrier for poison gases.”

Harry had to stop briefly for breath, and into that pause Professor Quirrell said:

“That’s three. You need ten. The rest of the class thinks that you’ve already used up the whole contents of the classroom.”

“Ha! The floor can be removed to create a spike pit to fall into, the ceiling can be collapsed on someone, the walls can serve as raw material for Transfiguration into any number of deadly things - knives, say.”

“That’s six. But surely you’re scraping the bottom of the barrel now?”

“I haven’t even started! Just look at all the people! Having a Gryffindor attack the enemy is an ordinary use, of course -”

“I will not count that one.”

“- but their blood can also be used to drown someone. Ravenclaws are known for their brains, but their internal organs could be sold on the black market for enough money to hire an assassin. Slytherins aren’t just useful as assassins, they can also be thrown at sufficient velocity to crush an enemy. And Hufflepuffs, in addition to being hard workers, also contain bones that can be removed, sharpened, and used to stab someone.”

DNA synthesizer, computers, machine shops, robots, chemistry and AI are not weapons. They are all greatly useful to improving our lives. The issue is that useful general purpose technolgies are useful specifically because we don’t know what people might think to do with them. Generality is what makes machines useful. At some capability level, generality also means ‘weapon’.

This is where my technology-libertarian instincts start fighting with the logic.

I don’t like where this goes

I want defensive technology to keep up. I don’t think it will work this way.

Governments will be inclinded for technology to become permissioned at some level. Some general purpose technology can’t be owned, or is severely locked down to the point we aren’t worried about local defense/offense. Maybe some general purpose technology is just outlawed. We have this today, but typically not for general purpose technology.

I hate this.

It conflicts with what I think has made computers and the internet useful. I don’t want governments deciding what software I can run, universal surveillance, or contraband knowledge. When electricity was first sold, nobody expected where we eventually took it. If it was heavily restricted for safety reasons we would today be in a much worse world.

But if Home Depot sells a $1,200 machine that prints objects using only dirt and air, and I can download printable drones and bombs, it is hard to argue against it being locked down.

Fragment the failure

Seeking a possible alternative: don’t require every person to be safe. Just limit how many other people one megaidiot can take with them.

Engineering does this everywhere. Buildings and networks have firewalls. Electrical systems have breakers. Trains have redundant safety circuits. We don’t generally make each component incapable of failing; we contain the failure.

Maybe humanity needs the same architecture. If one community can destroy itself without destroying everybody else, humanity is much more robust. This closer to the way it used to be in history… but we’ve been unifying the world’s civilization at the same time as giving everybody in it more power. We make transportation and communication cheaper, standards universal, software interoperable, and knowledge available everywhere. All of those are good individually, but they remove the firebreaks of communication over distance.

The sci-fi version

Geographic separation may eventually stop helping very much. I have a crackpot theory that the speed of light only exists because, if it didn’t exist, there is some kind of information or pattern that is antithetical to life. Any universe without a speed of light runs for awhile until the whole thing just instantly breaks once someone anywhere stumbles upon that ’end pattern’. So the anthropic principle dictates that we ‘of course’ need to live in a universe that is fragmented via some universal constant.

If a dangerous thing requires a 12-ton machine that only exists in New Jersey, putting people is sufficient fragmentation for humanity to survive. If the dangerous thing is a 1GB print file, Mars isn’t safe.

Taken far enough, I think this starts looking like some of the stranger science fiction around hostile memes, smart matter, grey goo, or civilizations that avoid about answering messages from other civilizations. The dangerous thing is the information needed to make the actual dangerous thing.

Actual isolation might require air-gapped distance (with restricted matter transfer), incompatible systems, and limited communications. I hate this too… humanity spreads off Earth, and we purposefully can’t call home.

It isn’t AI

I am concerned about AI for the reasons I’ve written about elsewhere. A sufficiently capable ASI is dangerous, if only because it’s a stand in for mythical geneies. Somehow solving AI doesn’t fix the broader issue.

We keep making technical expertise cheaper and easier to access, because that is what technological progress is, and it’s what a capitalist economy requires.

AI is just first technology that is forcing society to consider this general problem with General Purpose Technology.