Magic OR Good Engineering

Marc Andreessen from a16z wrote, or rather proclaimed, in 2011: “Software is eating the world.” The proclamation was prescient for the world of consumer software at least. There are very few aspects of digital consumer’s life untouched by the magic of software. You can feel it viscerally: when one tap on smartphone can summon a car; when screens recognize our faces; when late-night purchases show up at our doorstep in the morning.

That said, is there an appropriate reflection on “software eating the world” in 2021? The answer, that I have come to like, is that we have never been more capable in expressing world’s problems in software. This expanding leverage of software, seemingly infinite, has a flip side.

Which is, we have had no advances that have made it any easier to build good software systems. Anyone who has been in the trenches building software intuitively knows this.

The magic isn’t guaranteed. It can be only realized through good engineering.

The great computer scientist Fred Brooks saw this even before the current software revolution had started. Back then he described a timeless “essence” of building software. He divided this essence into two categories: “essential difficulties” and “accidental difficulties”. A brief example to review what he meant follows.

Say, you want to build a digital health app. To start, you need to understand the disease and its impact on patient’s life. You have to determine how to close the gap on existing care workflows, or if there is a need to redo them. You have to design a user experience, simple and compelling, to facilitate adoption. All of these details have to be represented as “highly precise” and “richly detailed” specifications for building the software. “Essential Difficulties”.

Then, there are other pieces to the puzzle. You have to make technology choices for the actual software system. You have to setup operations to build, manage and run the system. Last, but probably the most underappreciated thing, you have to build teams from a highly scarce talent to create and maintain the system. “Accidental Difficulties”.

As noted earlier, while we have become more capable with software, the essential never got, and as Brooks suggested, never will get easier. And the accidental, even though solvable, has simply gotten out of control. I mean, there are multi-billion dollar companies just addressing subsets of the accidental, e.g., Atlassian, all the X-aaS platforms.

In his 1986 paper, Brooks concluded he sees “no silver bullet” in the foreseeable future. Boy, was he right! So, if some of the consumer software examples I mentioned seem magical, it is only because the companies behind those experience empower talent to manage the essential, and spend a lot of capital to eliminate the accidental. There is simply no magic. It is as fundamental a truism as any in the world of software.

To borrow from Robert Heinlein, “one man’s magic is another man’s engineering”.

Andreessen had predicted healthcare and education sectors as the ones that will be “eaten” by software next. A decade hence, and with the benefit of the hindsight at our disposal, we can say that hasn’t happened yet. One thing we do know, software may eat the world but the world doesn’t just go away. The world doesn’t lose the ability to surprise us – especially where software systems have to interface with humans.

As Brooks explained decades ago, there is “no silver bullet”. There are only “silver linings”. And, the timeless principles of good engineering should be at the top of any list of such silver linings.

Creating software has to be a deliberate systems engineering exercise, not just indiscriminate deploying of technology to the current way of doing things. Solving complex problems with software is still an art, not science.

Above all, we have to know that true value comes from solving the essential, not from the accidental. Because it is the people who have to get healthy in the end. Apps can only be the means.

Photo by Shane Rounce on Unsplash