FATAL ABSTRACTION

An insider reveals the dark secrets of the software industry.

Digital technology was once touted as the path to a world that was easier, safer, and more connected. But at some point, says the author of this intriguing book, it became more part of the problem than part of the solution. Software is creeping into every corner of our society, and the trend shows no sign of slowing. Campbell’s career spans the gamut of the digital business, from gung-ho startups to ruthless tech giants, and he draws on his experience to compile a long list of stories of software going wrong. He discusses driverless cars, renegade dating apps, and near crashes of passenger aircraft. But the real issue is the interaction of software with what Campbell calls “managerialism,” or the desire of executives to increase revenue and reduce outlays. In fact, most corporate bosses have little understanding of how software works and see only the opportunity for speedy growth. Facebook, Google, Amazon, and Microsoft have already wound up or drastically cut back their internal offices responsible for ethical oversight, to save on staff costs. The problems have begun to multiply with the proliferation of AI systems, which could eventually take humans out of the equation entirely. Campbell believes that most software engineers have a genuine desire to make the world a better place, and if they work together they might be able to leverage their skills to introduce crucial safeguards. This sounds like an over-optimistic view, but Campbell is right in saying that change is needed. Without a new direction, he concludes, software will be “a digital straitjacket in which we bind ourselves ever more tightly, even as its embrace suffocates us.”

A disturbing look at the evolution of digital technology and the implications for society.