DON'T LET BOB DO IT

Published in slightly different form in Tomorrowsf (http://www.tomorrowsf.com), No. 16 (July-August 1999) (the last issue).

I teach a course in "Legal and Ethical Dimensions of Technology." The focus is computers, though of course technology is a much broader area than that and legal and ethical concerns hardly stop at the edge of the chip. And while the law itself may change from computers to genetic engineering to telecommunications and so on, the basic ethical issues--among them ownership, privacy, self-determination, and responsibility--remain much the same.

How do we decide such issues? Most folks ignore them, let them slide, go along to get along. Or they let a rule book such as the Bible tell them what to do. Or they simply do what their boss or lawyer tells them.

On the other hand, the Association for Computing Machinery (ACM), one of the major professional associations for people who work with computers, plays ethical hardball. Its Code of Ethics says that you have the responsibility and the right to make your own calls on right and wrong, and this is not optional. Your criteria may be religious, utilitarian, deontological, whatever. But if the boss tells you to do something you think is wrong, you have a duty to yourself, your profession, and the ACM Code to say no. You have to "know and respect existing laws," but "existing laws and rules may be immoral or inappropriate and, therefore, must be challenged." If the law tells you to do something you think is wrong, in other words, you have the same duty to say no. Is there a price to pay? Will you be fired or fined or jailed? You are responsible for the consequences of your decisions. If you fail or refuse, not only are you at the mercy of those who hold power in society--whose interests are by no means necessarily the same as yours, or society's--but you must accept those consequences as well. Ditto if you make the wrong call. Your choice, your lumps, and the important thing is to do what you think is right.

There isn't much comfort in that kind of ethical code. It's not for whiners. You're not allowed to blame anyone else, nor to delegate the work of making your choices for you.

Which brings me to the point of this column: People blame ("The devil--the boss--the system--made me do it") and delegate (to legislatures and churches) their ethical and moral choices so willingly that it bemuses me to note objections to science and technology on the grounds that they deprive us of choice. Protestors object to the study of the genetic roots of behavior on the grounds that to know the truth might threaten our belief in free will (Little Willy killed his sister, but he's no blister for it just means he's got bad genes). They object to science itself on the grounds that it threatens our sense that the universe is mysterious and wonderful (we can no longer believe in ghosts, goblins, gods, angels, and fairies). They object to the telephone because it gives us no say in who we admit to our homes, to the Internet because it forces pornography upon us willy nilly, to.... You get the idea.

Science is surely a greater threat to choice than is technology, for once you know the truth, you can no longer choose to believe falsehood (except in the spirit of let's pretend, though politicians may make you wonder). As for technology, well... one can always choose to unplug the telephone, or to turn off the ringer, or not to answer it, or not to have one at all. Internet pornography is a nonissue, for no one ever has to see a single unwanted picture. Surf with your images turned off; turn them on only when you know you're on safe ground. Don't blame the technology because you're too lazy to be cautious.

If it is only poor carpenters who blame their tools, then we're lousy carpenters. We have a powerful tendency to blame technology when we think it fails us. We blame the disasters at Love Canal and Three Mile Island and Chernobyl and Bhopal on the technologies involved, when in each case the real culprit was human error. We blame our computers for losing files when the real problem is that we neglected to specify a directory (or folder) or we got lazy about backing up our work. Sometimes this makes us look really stupid, as when we call the help desk because our computer won't work, and the very first question they ask is, "Is it plugged in?"

Once upon a time, we projected will and intent (moral agency) on trees, rocks, storms, and the like (the rock that punctured your oil pan the last time you drove out that dirt road to camp did it on purpose, you know). Today we do the same thing with computers, aided by efforts to make these machines interact with us roughly the way we do with people. Researchers are even developing robots with faces to facilitate "meaningful social exchanges with humans."

My Turbo Tax software already has an avuncular video accountant that will talk me through the process. Children's educational software often has an embedded coach of a similar sort. And I'm pretty sure that at least some people talk back to these figures, plead with them, cuss at them, and otherwise treat them like people. This isn't even new; Weizenbaum's "Eliza"--developed in the sixties to mimic a Rogerian psychotherapist (who interacts with patients in a very limited way)--can be very convincing; Weizenbaum's own secretary would hold long, intense conversations with the program. You can, too.

The tendency to trust the machine has been a major factor in limiting applications of medical expert systems. Consider the APACHE system, originally designed in part to evaluate critically ill patients and say when to withdraw life support. Few would be willing to set it up as an automated plug-puller, but even when it is used only to analyze and advise, medical personnel may relieve themselves of considerable anguish by saying, "The machine said it was time to let the patient go." Since no one would be surprised if a member of the patient's family disagreed and filed a liability suit, the makers of such systems have tried to distance them from actual vital decision-making, selling them as "healthcare decision support systems."

Some people urge that we do our best to design computers in such a way that they do not lead us to think of them as having the power to make decisions for us. We should do away with video images and voice tracks and stick with mechanical imagery. However, there will still remain the human tendency to anthropomorphize the machine, to ascribe to it personality and will and power to command us. After all, people were anthropomorphizing computers long before anyone ever thought of making them "user-friendly." We also don't have much trouble naming our cars or cussing out a broken washing machine.

Even if we call the car "Lassie," we can't tell it to go run an errand for us. But with computers that's just what's coming soon to a screen near you. A great deal of work is going into "software agents," autonomous programs that can indeed run errands for you. Want a book or a CD? Don't waste time checking Amazon.Com, Barnes & Noble, and any other on-line bookseller you know of. Pull up your "Purchasing Agent," tell it what you want, and it will do the job for you, find the best price, place the order, and return with a jaunty salute and a "Mission accomplished, Sir!"

I'm not quite kidding about the salute. These things will come with faces and expressions; they'll present themselves on your screen as people, and it won't be hard at all to think of them that way. Since they will be autonomous, using them will involve a genuine delegation of decision-making power, of precisely the sort that concerns the critics of technology. And they're hardly limited to buying books and CDs; the MIT Media Lab has a long list of agent projects. Some will cruise the Web to find information for you, check chat rooms to look for interesting conversations, prepare news summaries, hunt down experts, remember things, sort and/or answer email, and much, much more. See also the Swedish Institute of Computer Science's Agents index. The University of Maryland's Laboratory for Advanced Information Technology also offers a great deal of information on the topic.

Is there any danger that software agents will take over our lives? I'm sure that some critics will see them as a dangerous extension of long-standing trends. Delegating our decision-making to them will be convenient, but it will mean surrendering control and autonomy, putting ourselves at the mercy of a machine whose agenda may not suit us (yes, agents are described as having their own agendas; take another look at this site). They could get us into all sorts of trouble if they went wrong--spending us into bankruptcy, ignoring crucial email, or failing to include vital information in a news digest. They could even cause problems if they didn't go wrong--tell a data-seeker to go to work, and watch the cop on the doorstep arrest you for industrial espionage.

Yet I am somehow not terribly worried. People do delegate their choices and responsibilities, but they also want to maintain control. They draw limits. I despise that Turbo Tax video accountant; the very first thing I do when I get each year's version of the program is figure out how to bypass it. (I bet you do too!) I also despise word processors that complete words and sentences for me or nag me about spelling and grammar.

And some attempts to invite people to delegate more to the software in their computers have been notable failures. Remember "Microsoft Bob"? Released in 1995, it offered users a set of cartoon-character "guides" designed to make home computers seem friendlier and less intimidating. It was named by an advertising agency to reflect the "social and friendly nature of the software package."

Some people liked it. FamilyPC magazine's testers found it easy to use and helpful despite being slow and nonintuitive. Some did not. The Boston Globe called "Bob ... an utter simpleton, poorly designed and dreadfully executed." Some made fun of it. So did some more. And Bob vanished.

The reason for that vanishing may actually offer us some hope. Bob was cutesy and clumsy, but it also denied the user a sense of control over the machine. Users seem to prefer to push the buttons themselves, and in a free marketplace their choices of what to purchase will reflect that. They may not want to go all the way back to the dreaded A:> prompt, but they don't want a machine that does everything for them. Therefore, despite the tendency of the computer industry to keep devising nifty ways to make the computer a friendly, social, autonomous, independent agent, most folks won't buy it.

We may be lazy, but we're not that lazy. We may even not be lazy enough to accept automated highways once we have the ability to build them. We'd rather do some things ourselves.

And that suggests to me that those critics of technology who decry the loss of choice are missing an important point: They're wrong. People insist on preserving the ability to choose, to control, to do things themselves. They embrace technology that increases that ability, as a car permits a city worker to live in the suburbs and choose gardening, walks in the woods, and so on, over stoop-sitting. Technology that, like Bob, fails to increase choices dies.

Perhaps ethical codes that insist we make our own choices between right and wrong, and take responsibility for the consequences of our choices, have a chance after all.


Dr. Thomas A. Easton is Professor of Life Sciences at Thomas College in Waterville, Maine. He has been the Analog book columnist for the last twenty years. His latest novel, Unto the Last Generation, is available only on-line, from Mind's Eye Fiction (http://tale.com). Last year's Silicon Karma (White Wolf, 1997) was well received; copies are now available from the author. His latest nonfiction books are Taking Sides: Clashing Views on Controversial Issues in Science, Technology, and Society (Dushkin Publishing Group, 1995, 2nd ed., 1997, 3rd ed., 1998) and Periodic Stars: An Overview of Recent Science Fiction (Borgo Books, 1997).

Previous columns are available here.