Bugs!

By Jeff "nonick" Solomon

That we’ve become so accustomed to buggy software is testimony to the fact that the computer industry in general is still at a very underdeveloped and frontier-like phase of existence. More established industries, like consumer electronics and automobiles, operate in dramatically different ways in regard to product failures.

For one, there are far fewer instances of "buggy" radios, toasters, pens, or even cars. This is not to say that products in other industries don’t malfunction or always ship in perfect condition. There are problems with nearly every complex product on the market. However, these industries typically take a more extensive and comprehensive approach to quality assurance, and operate on timetables that allow them to take more time to market for their products.

In the computer industry, where release dates are often set ambitiously and continuous upgrades are the rule, there’s often less time to exhaustively test a given title. It’s not that quality assurance is not taken seriously, but rather, the basic nature of software is fluid. While a team of engineers working on a new razor blade can sit back and relax for a while once the product has been completed and sent to market, a software release is little more than the process of officializing and distributing a given version of source code. Once this happens, work often continues on the product; new features are added, problems are fixed, and so forth. There is, in other words, no single "final product."

Plus, in this age of growing Net connectivity and user-savvy, companies often assume that, if need be, they can distribute patches for buggy software. For most other product types, this option doesn’t exist. A malfunctioning product can either be repaired by a technician or recalled. Toaster heater element 1.01, which fixes the spontaneous neutering problem with 1.0, will never be a reality.

So, what exactly is a software bug? It could be the result of a programmer’s typo; a glitch that occurs when software is forced to run on hardware that isn’t fully (or properly) supported; a problem that arises from poor software design in general; or a zinger that pops up because designers failed to ask enough "what ifs?".

All of these scenarios count as bugs. Depending on who you talk to, you’ll get a different answer as to a technical definition. Many developers, for example, consider the phrase "Y2K bug" to be a misnomer because the problem arises from an intentional decision to use the two digit date system, as opposed to four. While this scenario can wreak havoc when calculations extend beyond the year 2000 (or when a variety of other situations occur), many people don’t consider the problem to be a bug, per se, because it isn’t an unintentional failing on the behalf of the developer. If, however, the Y2K problem was caused by a compiler glitch, or something along those lines, then it would classify as a bug according to this technical definition.

To most people, however, it doesn’t matter what is causing the problem. Basically, from a user’s perspective, anything that prevents a program from running the way it’s supposed to can be classified as a bug.

Likewise, regardless of cause, anything that causes a program to malfunction needs to be fixed. I spoke with designer Harry Teasley of Valve Software about the issue, and he had some interesting things to say. My first line of questioning revolved around the mechanics of dealing with bugs if and when they appear. Basically, I wanted to hear about the process that developers go through to test for, isolate, and fix bugs. Harry told me in a very clear, concise manner. And now, he’ll tell you, too:

 

...bug finding is an everyday thing in all game companies. People are constantly playing the game, as new content is added, and in this constant playing, they hit bugs. They get annoyed. A bug database gets added to, and the people responsible for fixing the bug go through the database, and do work in order to claim they fixed the bug. The bug is then sent back to the person who reported it, for them to verify whether or not it was indeed fixed.

This happens a lot. At a point somewhere within sight of the project deadline, product features are locked, and the bug fixing begins in earnest. It becomes the main focus of the development team, who divide their time between playing the game looking for things to fix, looking for things that were supposedly fixed, and fixing things. The bugs are prioritized, and the Class A bugs, the crash bugs or progress-stopping bugs, are fixed first. The rest are prioritized according to a variety of factors, and are fixed as they can be.

Me again. The basic development process works almost like a building construction project. The very early stages involve raw materials that in no way resemble an actual building. Concrete, glass, plastic, and so forth. Then, the fundamentals are put up: the basic structure, without windows, doors, wires, or toilets. Once the "skeleton" is up, the construction workers can actually move about inside the house and work on it from a variety of angles. While it may not be pretty, you can start to see the form of a building taking place. Eventually, after a good deal of poking around, the building is finished.

Software development works in much the same way. A game begins as a collection of parts that are constructed individually. For a while, the "game" remains solely a collection of files that are slowly built up to a point where they can finally be connected and executed in a rudimentary fashion. At this point, the primary program files can be linked and, theoretically, the game "exists" in a basic, rough form. The developers can "play" the game, to some extent, and begin to iron out problems and push it toward completion. From here on in, they’re poking and prodding, trying to add polish and smooth out the rough edges.

There isn’t a magic moment when a software project is "finished." Rather, there is a period of time when the developers consider the program to be "feature complete," and focus their time solely on trying to find problems that they had not anticipated. This process is especially important for console games, which can’t be patched. If a Playstation or Dreamcast game slips out the door with some serious problems, there is little a company can do to solve them. Once again, Harry Teasley:

On consoles, bugs are much more of a pain, but they're also easier to handle. The game only runs on one kind of hardware, and doesn't have to share with any other programs, so you really know exactly what you're dealing with. Compared to the uncertainty of hardware configurations in the PC world, it's a paradise.

Bug fixing is more intense, though, because once a game is submitted for testing at the console manufacturer, it had best not have any bugs or the consequences are more dire than on the PC. Games enter a queue, and get rigorously tested for 90 days or so, and if anything is found that needs to be fixed, it gets kicked out of the queue and has to be resubmitted, and start the 90 days of testing all over again, regardless of how far it got the first time through. A perfectly rational system, but it can also be very nerve-wracking, and expensive.

That kind of zero-tolerance for bugs attitude can only work on platforms where the hardware is invariant and there is no ability to self-publish software (the console manufacturer approves all games for publication, and controls production.)

 

So, if you’ve ever wondered why it seems like console games are less buggy than their PC cousins, now you know. They test those games more extensively, and the developers have fewer variables to work with in regard to compatibility.

 

Some day, hopefully, the PC landscape will advance and simplify a bit to the point where game developers don’t have to fret as much about the "Wild West" of system configurations and hardware interfaces. This panacea may come in the form of a more streamlined approach to operating system design, or in some other initiative brought about by the recent onslaught of open source "let’s make PCs work right, already" projects.

 

Until then- and even after something like that- we’ll have to live with the fact that almost all software is going to have bugs. The real solution lies in the approach developers and distributors take to fixing them. As the computer industry matures and advances, it’s inevitable that companies will be held more accountable for flaws that are found in their products. Most people are simply unwilling to accept the fact that software bugs are commonplace, and many don’t have the patience or means to patch their software.

 

At the same time, software distribution will continue to advance to the point where most programs are delivered- and even maintained- automatically through the Internet. The Windows Update feature in Windows 98- for all of the flack it gets- is a good example of initiatives that are being taken to create software that is "self-healing." Eventually- hopefully- bugs will be fixed automatically.

 

On a final note, I’d like to cut through the gloom and doom and bring Harry back to comment on one of his "favorite" bugs. Fortunately, not all bugs are evil:

 

A favorite bug of mine was in a Harrier flight sim MicroProse did several years ago: you could hover upside down and drop bombs into space. There was some hemming and hawing about how "we don't need to fix that, no one would ever try to do it, anyway," but I believe more rational folks prevailed. One thing you learn very quickly is that the things you are certain players will never discover get reported on the fan pages within 48 hours of the game hitting the stores.

We all know the truth in that.

- Jeff Solomon is a regular contributor to loonygames.