In 1995, computer scientist Niklaus Wirth published the article “A Plea for Lean Software” in the magazine Computer. Its thesis seems to suggest that AI based on neural networks may be doomed to fail, not because of a lack of electrical energy, but because of software inefficiency.
The experience of an admittedly unusual user
This month, I got clobbered by the content management system (CMS) I use for my blog. The company that runs the CMS had been pushing me for years to accept its more powerful version. I didn’t want it. The old version took care of my needs. It was actually a bit too complicated already, but it was a lot simpler than the new beast. More power would only make things worse. With a more powerful system, I would have to work harder to get the same job done, with no improvement to show for the effort.
They asked me what I wanted, and I told them. But they converted my operation to the new version anyway, providing no possibility that I could keep doing things as I had done. Now I can change default typefaces for individual paragraphs. But why would I do such a thing?
The change to greater power came with other changes, some of which were just as I had feared. One was a computer bug unlike anything I have ever seen before, and I have worked with computers since about 1968. I will use it as an example.
The CMS is so difficult to work with that I started editing text for posts in a word processor instead of using the posting software itself. But when I copy a paragraph from the word processor into the post, a carriage return got inserted at every point that had been edited. What had been a five-line paragraph might take more space than could fit on a screen.
What kind of software would do that? Certainly not any kind I want. I can get rid it by finding a new CMS, but that would be a monumental pain. So I am left trying to figure out which would be less difficult, switching to a new system or keeping the one I have been using for fourteen years, now that it behaves like nothing I have ever seen before.
I went on. I started thinking about past experiences with computers. My first personal computer was a TRS-80 Model 3. It could keep up with me, no matter how fast I typed. Of course, it didn’t have a word processor, but it could keep up.
Next I got an IBM PC. At first, I did programming using extremely simple editors. The first actual word processor I got for it was pfs:Write, in 1983. It was the only word processor I have ever heard of with documentation that started with a sentence that went something like this: “If you have ever used a word processor before, you probably don’t need to read this.” It was simple. I could have written large works of fiction with that word processor, but just one chapter at a time.
Now I use LibreOffice. It is powerful and sophisticated. It’s free. I can handle huge files with it. It does just about everything I want and a host of things I don’t want. Menus are long and I have to search for items in them. Its complexity does slow me down, though honestly that is only marginal. I can load an entire novel into LibreOffice and tell it to use the ligatures for the Times Roman font.
But that is hardly a priority.
What I was finding was that computers are getting more difficult and slow, even as they get more powerful. I started formalizing my research on the subject. And then, I discovered I was not the first to look into it. Niklaus Wirth had done the same thing.
Dr. Niklaus Wirth was a well known Swiss computer scientist. He developed the Pascal programming language I had used in days of yore. He is thought to have developed what is called “Wirth’s Law,” though he gave credit to someone else. He did circulate it. An article about it in Wikipedia starts out with the sentence “Wirth’s law is an adage on computer performance which states that software is getting slower more rapidly than hardware is becoming faster.”
Software slows things down faster than hardware speeds them up!
The implications of this for AI might be profound. Another Wikipedia article, “Software bloat,” goes a bit further. The implication is that the more the software is doing, the faster the hardware has to be to keep up. But the hardware can’t keep up, because the software is faster at slowing things down. This means that from the human point of view, things just keep getting more bloated by features, more confusing, and more sluggish.
I think there could be a simple flaw underlying AI based on neural nets. It is bloated. The amount of hardware needed is gigantic and expanding faster than data centers can grow. Perhaps that could be why some experts want to build nuclear power plants just to support AI. Maybe, but the time the data centers are operating, a single nuclear reactor would not be enough.
Actually, why not three?
Because of software bloat, the AI system could slow down, and I think it probably will. Of course, when that happens, the easy solution may be to make the hardware more powerful. All you really have to do to speed up hardware is to increase the clock speed. Okay, I admit it is a little more complex than that. But with software, any work you do is likely to make everything more complex, and that means slower. And sometimes you don’t know why.
My Conclusions
We are investing a lot into a form of AI that uses a lot of computing power, consuming a lot of energy, to produce results that we can see are sometimes wrong and sometimes worse. The computers that do this are so big they go past requiring their own buildings, reportedly to the point of requiring their own nuclear power plants.
In terms of bloat, a nuclear plant is about 35% efficient, with the rest of the energy it produces lost to cooling. Out of the 35% of its energy that actually goes to the data center, a fair amount is used just cooling the data center itself. And of course, some of the energy goes into cooling the office, including AC for the guy who throws the “On” switch to start the system running, and more AC for the guy who throws the identical-looking “Off” switch.
That is a lot of waste, but it could be just a beginning. What happens if the whole thing turns out to be an investment bubble? The amount being invested in AI is enough that if it fails, it could cause very serious economic problems for the US and worldwide.
AI is a technology that needs oversight. And it needs a thorough examination of issues and solutions. Also, it is really important that it get oversight before the money is spent.