How Von Neumann Showed The Way

There were ten people in the party, and they were about to descend into the Grand Canyon. The guide wore a cowboy hat and leather chaps; a coil of rope hung from his saddle. Most of the others were dressed for a day outdoors, wearing hats, loose-fitting shirts, and the like. At the rear of the group was John Von Neumann—hatless and in the formal suit and tie of a banker. Moreover, while everyone else sat on a mule facing right, his faced left.

Von Neumann had been following his own rules for years. He owned a photographic memory that held the complete texts of works of literature and one of the world’s largest collections of off-color limericks. Yet he would phone home to ask his wife to help him remember an appointment. He loved to throw parties—and sometimes would steal away to work in his office while his guests remained downstairs. Among his friends he was nearly as well known for his traffic accidents as for his accomplishments in mathematics. A strong supporter of the military, he was fond of attending nuclear-weapons tests. He died of cancer at the age of fifty-three.

Through it all, he was one of the century’s most creative and productive mathematicians, lifting his intellectual scepter across a host of technical fields. Mostly he worked with pencil and paper, but in the years after 1945, for the first time in his life, he set him- self the task of managing the design and construction of a piece of equipment. This was the Institute for Advanced Study computer, and it set the pattern for subsequent computers as we know them today.

What distinguished this IAS machine was programmability. It embodied Von Neumann’s insistence that computers must not be built as glorified adding machines, with all their operations specified in advance. Rather, he declared, they should be built as generalpurpose logic machines, built to execute programs of wide variety. Such machines would be highly flexible, readily shifted from one task to another. They could react intelligently to the results of their calculations, could choose among alternatives, and could even play checkers or chess.

This represented something unheard of: a machine with built-in intelligence, able to operate on internal instructions. Before, even the most complex mechanisms had always been controlled from the outside, as by setting dials or knobs. Von Neumann did not invent the computer, but what he introduced was equally significant: computing by use of computer programs, the way we do it today.

The roots of this invention lay not in electronics but in the higher reaches of mathematics, in a problem that tantalized specialists in mathematical logic during this century’s early decades: the challenge of establishing basic foundations for math. These would take the form of an explicit set of definitions and axioms, or fundamental statements, from which all known results might be derived.

Everyone expected that such foundations could be constructed if people were only clever enough. David Hubert of Göttingen University, widely regarded as the world’s leading mathematician, summarized this viewpoint in a 1900 address: “Every mathematical problem can be solved. We are all convinced of that. After all, one of the things that attracts us most when we apply ourselves to a mathematical problem is precisely that within us we always hear the call: here is the problem, search for the solution; you can find it by pure thought, for in mathematics there is no ignorabimus [we will not know].”

In fact, however, a powerful ignorabimus lay at the center of the problem of mathematical foundations. The man who demonstrated this was Kurt Goedel, a logician at the University of Vienna. He was a smallish man with an earnest expression and a thick pair of glasses; he appeared even smaller than he was because of his reluctance to eat. Psychological depressions and other illnesses dogged him throughout much of his life, made more serious at times by his distrust of doctors. In contrast with the gregarious and hearty Von Neumann, Goedel was solitary in his habits, but he did form a few close relationships. One was his lifelong marriage to Adele Nimbursky, a former cabaret dancer. Another was a warm friendship with Albert Einstein.

In two epochal papers, published in 1931, when he was twenty-five, Goedel showed that no foundations could be constructed. More particularly, he showed that if anyone tried to set forth such foundations, it would be possible to devise mathematical statements that were “formally undecidable”—incapable of being proved or disproved using the proposed foundations. Anil Nerode of Cornell University describes this conclusion as “the paper that everyone read, because it was the most signal paper in logic in two thousand years.”

In particular, this work offered two major results for the eventual development of computer science. To prove his theorems, Goedel introduced a notation whereby statements in mathematical logic were encoded as numbers. Every such statement could be expressed as an integer, usually a very large one, and every integer corresponded to a statement in logic. This introduced a concept that would be key to the later advent of computer programming: that not only numerical data but also logic statements—and by extension, programming instructions—could be expressed in a common notation. Further, Goedel’s work showed that this notational commonality could give results of the deepest significance in mathematics.

Among the mathematicians who soon took up the study of these matters was Alan Turing, of Cambridge University. Turing was a vigorous man, fond of running and cycling, and sometimes eccentric. Issued a gas mask, he wore it to prevent hay fever. Fearing that British currency would be worthless in World War II, he withdrew his savings and purchased two ingots of silver, buried them in his yard—and then failed to draw a suitable treasure map that would permit him to find them. And when his bicycle developed the habit of having its chain come loose, he refused to take it in for repairs. Instead he trained himself to estimate when this was about to happen so he could make timely preventive fixes by himself.

Turing was a twenty-five-yearold undergraduate when he made his major contribution to computer science. It came in a 1937 paper, “On Computable Numbers,” in which he specifically dealt with an imaginary version of the computer. This idealized machine was to follow coded instructions, equivalent to computer programs. It was to deal with a long paper tape that would be marked off in squares, each square either black or white and thus representing one bit of information. On this tape, in response to the coded commands, the machine would execute a highly limited set of operations: reading, erasing, or marking a particular square and moving the tape.

Analyzing this idealized computer, Turing proved that it offered properties closely related to Goedel’s concept of formal undecidability. What was important for computer science, however, was another realization: that with sufficiently lengthy coded instructions this simple machine would be able to carry out any computation that could be executed in a finite number of steps. Here, in its essential form, was the concept of a general-purpose programmable computer.

The basic idea of a calculating machine was not new. The first crude adding machines dated to the seventeenth century. In the nineteenth century Britain’s Charles Babbage, assisted by Lady Ada Lovelace, had struggled to invent an “Analytical Engine” that was really a crude mechanical computer. What was new and pathbreaking in Turing’s work was that for the first time he gave a clear concept of what a computer should be: a machine that carries out a few simple operations under the direction of a program that can be as intricate as one may wish.

These developments were very interesting to John Von Neumann. As a student in Germany (he was born in Hungary in 1903), he had worked closely with Hubert himself, plunging deeply into the search for mathematical foundations. He had shared Hubert’s belief that such foundations could in fact be constructed, had written a paper that contributed some mathematical bricks to the intellectual masonry—and was surprised and chagrined by Goedel’s proofs. He had not thought that formal undecidability might exist, and he came away with the feeling that Goedel had scooped him.

He had plenty of reasons to feel confident, however. The son of a Budapest banker who had received a minor title of nobility, the source of his “Von,” he had shown himself very early to be a Wunderkind , dividing eight-digit numbers in his head at age six and talking with his father in ancient Greek. By age eight he was doing calculus and demonstrating a photographic memory: he would read a page of the Budapest phone directory and recite it back with his eyes closed. His father’s wealth made it easy for him to attend the University of Budapest, from which he traveled widely: to the University of Berlin, to Zurich and its equally famous university, and to Göttingen, the world’s center of mathematics. At age twenty-two he received his Ph.D. Nor did he keep his genius to himself; his daughter Marina, born in 1935, would rise to become a leading economist in the United States.

Von Neumann had made his reputation during the 1920s, establishing himself as clearly one of the world’s outstanding mathematicians. Particularly significant was his work in developing a rigorous mathematical basis for quantum mechanics. That brought him an invitation to Princeton University, which he joined in 1930, when he was twentysix. “He was so young,” says a colleague from around that time, “that most people who saw him in the halls mistook him for a graduate student.” Then in 1936 Turing came to Princeton to do his graduate work; he was twenty-four. Von Neumann, who had moved to the Institute for Advanced Study in 1933, was quite interested in Turing’s work and offered him a position as his assistant after he received his doctorate, but Turing chose to return to Cambridge.

Meanwhile, Von Neumann was doing much more than reading his colleagues’ papers. During the early 1940s he began to work extensively on problems of fluid flow. These problems were widely regarded as nightmares, marked by tangles of impenetrable equations. To Von Neumann that meant they were interesting; understanding them could lead to such consequences as accurate weather prediction, and because such problems posed intractable difficulties, they were worthy of his attention.

Then came the war and the Manhattan Project. Von Neumann’s expertise in fluid flow now took on the highest national importance. As the work at Los Alamos advanced, he became responsible for solving a problem that was essential to building the plutonium bomb. This was to understand the intricate physical processes by which a thick layer of high-explosive charges, surrounding a spherical core of plutonium, could detonate to produce an imploding shock wave that would compress the core and initiate the nuclear explosion.

As his colleague George Kistiakowsky later wrote, high explosives had been “looked upon as blind destructive agents rather than precision instruments.” In the plutonium bomb, however, it would be essential to predict with some accuracy the behavior of the shock waves that would converge on the core. Even Von Neumann’s brilliance was inadequate for this. He had hoped that ingenuity and insight would enable him to simplify the pertinent equations to a form both solvable and sufficiently accurate. His collaborator Stanislaw Ulam insisted that it would be necessary to face their full complexity and calculate them, in an age when there were no computers, using methods that would later be programmed to run on computers. Fortunately, the Los Alamos lab was due to receive a shipment of IBM calculating machines. Stanley Frankel, another Los Alamos man, set up a lengthy sequence of steps that these machines could carry out, with Army enlistees running them. It amounted to a very slow computer with human beings rather than electronic devices as the active elements, but it worked. Von Neumann got the solutions he needed, and he proceeded to design the high-explosive charges for Fat Man, the bomb dropped on Nagasaki.

Meanwhile, at the University of Pennsylvania, another effort as secret as the Manhattan Project was under way: the construction of the first electronic computer. This was ENIAC (Electronic Numerical Integrator and Computer), an Army-sponsored project intended for use in calculating the trajectories of artillery shells. Its employment of vacuum tubes rather than people as active elements represented a decided advance, but while the potential value of such tubes for high-speed computing was widely appreciated, the tubes of the day were not particularly reliable. That did not matter when only a few were needed, as in radar or radio, but it would matter greatly in a computer, where a single failed tube could vitiate a lengthy calculation. (Because of this, Harvard’s Howard Aiken had gone to work on a computer that would use the electromechanical switches of telephone circuitry. They were far slower than vacuum tubes, but still much faster than human beings, and they were reliable.)

The ENIAC project leaders, John W. Mauchly and J. Presper Eckert, Jr., solved the reliability problem in a simple way. They were working with tubes whose manufacturers had guaranteed a service life of twenty-five hundred hours. With 17,468 tubes in ENIAC, that meant one could be expected to fail, on the average, every eight minutes—and with major computations requiring weeks of operation, this was quite unacceptable. Eckert, however, simply “unloaded” the tubes, arranging it so that they would handle no more than one-half of their rated voltage and onefourth of their rated current. This reduced the failure rate from one every eight minutes to about one every two days, which was sufficient for practical operation.

The Army’s representative on the project was Lt. Herman H. Goldstine, who had taught mathematics at the University of Michigan. He was working out of the Aberdeen Proving Grounds in Maryland, where Von Neumann was a consultant. One day in August 1944 he saw Von Neumann waiting for a train. “I had never met this great mathematician,” Goldstine recalled. “It was therefore with considerable temerity that I approached this world-famous figure, introduced myself, and started talking. Fortunately for me Von Neumann was a warm, friendly person who did his best to make people feel relaxed in his presence. The conversation soon turned to my work. When it became clear to Von Neumann that I was concerned with the development of an electronic computer capable of 333 multiplications per second, the whole atmosphere changed from one of relaxed good humor to one more like the oral examination for a doctor’s degree in athematics.”

ENIAC was a large air-conditioned room whose walls were covered with cabinets containing electronic circuitry—three thousand cubic feet of it. It weighed thirty tons and drew 174 kilowatts of power. Its computational speed and capability would fail to match the hand-held programmable calculators of the mid-1970s, but even so, it was such an advance over all previous attempts at automatic computation as to stand in a class by itself. Still, it was not without its faults, as its builders were well aware. Its main memory (random-access memory) could hold only a thousand bits of information—the equivalent of about three lines of text. And it was completely lacking in any arrangements for computer programming.

You did not program ENIAC; rather, you set it up, like many other complex systems. Although it was a generalpurpose computer, able to solve any problem, it relied on physical interconnections. You prepared for a particular problem by running patch cords between jacks and other plugs, with cabling up to eighty feet long. The task could take two days or longer. In a 1943 report the builders admitted that “no attempt has been made to make provision for setting up a problem automatically,” adding that “it is anticipated that the ENIAC will be used primarily for problems of a type in which one setup will be used many times before another problem is placed on the machine,”

By the summer of 1944, however, Eckert, Mauchly, and their colleagues were already beginning to think seriously about ENIAC’s successor. This would have the name EDVAC (Electronic Discrete Variable Automatic Computer). As early as January of that year Eckert had described a computer in which an “important feature” was that “operating instructions and function tables would be stored in exactly the same sort of memory device as that used for numbers.” Eckert was also inventing an appropriate memory device: a “delay line,” or long tube filled with mercury in which bits of data would take the form of pressure pulses traversing the tube at high speed. And in October 1944, at Goldstine’s urging, the Army awarded a $105,600 contract for work on the EDVAC concept.

Into this stimulating environment stepped Von Neumann. He joined the ENIAC group as a consultant, with special interest in ideas for EDVAC. He helped secure the EDVAC contract and spent long hours in discussions with Mauchly and Eckert. “He was really racing far ahead and speculating as to how you build better computers, because that’s what we were talking to him about,” Mauchly later said. “We said we don’t want to build another of these things [ENIACs]. We’ve got much better solutions in many ways.”

Von Neumann’s particular strength was the logical structure of a computer, the details of its logic operations. His leadership made the EDVAC discussions more systematic. Before his arrival Eckert and Mauchly had relied mostly on informal conversations; with Von Neumann there were regular staff meetings with recorded minutes. As Goldstine reported to his boss, Von Neumann was “devoting enormous amounts of his prodigious energy to working out the logical controls of the machine. He also has been very much interested in helping design circuits for the machine.”

In late June 1945, working at Los Alamos, Von Neumann completed a 101-page document titled “First Draft of a Report on the EDVAC.” In his clear and penetrating way, he set forth an overview of the design of a digital computer that would feature stored-program operation. It had much more than circuitry and logic; it reflected Von Neumann’s broad interests by drawing on the work of Warren McCulloch, a neurophysiologist who in 1943 had published a description of the functioning of the human brain. Von Neumann boldly drew comparisons between his electronic circuits and the brain’s neurons, emphasizing that just as the brain relies on its memory, so the computer would depend on its programs. Goldstine soon was distributing copies to interested scientists. In time the “First Draft” would become one of the most influential papers in computer science.

Goldstine circulated the draft with only Von Neumann’s name on the title page. In a later patent dispute, Von Neumann declined to share credit for his ideas with Mauchly, Eckert, or anyone else. So the “First Draft” spawned the legend that Von Neumann invented the stored-program computer. He did not, though he made contributions of great importance. But by writing the “First Draft,” and subsequent reports, he gave a clear direction to the field. The prestige of his name ensured that he would be followed. “The new ideas were too revolutionary for some… ,” said the British computer expert Maurice Wilkes. “Powerful voices were being raised to say that… to mix instructions and numbers in the same memory was to go against nature.” Von Neumann stilled such doubts.

As 1945 proceeded, he became convinced that he should not merely write about stored-program computers but should take the lead in another way: by building one. Raising money for such a project would be no problem; he knew his way around Washington. The problem was that his home base was, and had been since 1933, the Institute for Advanced Study, in Princeton, New Jersey. Founded by the department-store magnate Louis Bamberger and his sister, Carrie FuId, it was a center for pure contemplation and thought, a place where Einstein and Goedel would feel at home and spend much of their careers. To propose building a computer at IAS was like offering to install a radar facility in St. Peter’s Basilica.

Von Neumann overcame his colleagues’ objections by playing the IAS against two other institutions that wanted him, the University of Chicago and the Massachusetts Institute of Technology. At MIT Norbert Wiener, a colleague from Von Neumann’s G’f6ttingen days and himself a pioneer in computing, offered Von Neumann the chairmanship of the mathematics department, emphasizing that he would be free to work on his “favorite projects.” Chicago offered to set up an Institute of Applied Mathematics with Von Neumann as its head. Faced with such offers and wanting to keep Von Neumann as one of their own, his IAS colleagues gave in and granted permission, consoling themselves with the thought that the new computer might after all be useful in research.

Then the ENIAC group broke up. The source of this was a new director of research at the University of Pennsylvania, where the computer had been built, Irven Travis. Travis had spent his war in the Navy and proposed to run a tight ship now that he was back in the civilian world. He soon was quarreling with Eckert and Mauchly over the issue of patents. The two ENIAC inventors saw great commercial prospects in computers and had a letter from the university president that agreed they could hold patents on ENIAC. Travis, however, insisted that they must sign patent releases. He made no bones about it; in one meeting with Mauchly he stated, “If you want to continue to work here at the university, you must sign these agreements.” Mauchly and Eckert refused and were soon out on their own as independent entrepreneurs.

By the summer of 1946, then, three groups were seeking to build a stored-program computer along the lines of the “First Draft.” Eckert and Mauchly had by far the most experience in this area, but were out in the cold with little money, few contacts, and slight business experience. The remnants of the ENIAC group, at the University of Pennsylvania, had few good people but were committed by contract to build an EDVAC, and build it they would, however slowly. Von Neumann had the overall vision, the charismatic reputation, the genius, and the acquiescence of the IAS. What he lacked was experience in project management.

Of these deficiencies, Von Neumann’s was the most easily remedied. He had technical support from RCA, which had built a lab in Princeton. He had Herman Goldstine, who left the Army to join him. And at Norbert Wiener’s recommendation he hired Wiener’s wartime assistant, Julian Bigelow, who had worked on radar-guided fire control of antiaircraft guns and who knew how to build electronic systems of a very demanding character.

The computer was to be built in the boiler room of FuId Hall, the main building at the IAS. As Bigelow describes the work, “Von Neumann would put half-finished ideas on the blackboard and Goldstine would take them back down and digest them and make them into something for the machine. On the other hand, Von Neumann often had only the foggiest ideas about how we should achieve something technically. He would discuss things with me and leave them completely wide open, and I would think them over and come back with an experimental circuit, and then my group would test it out.”

When completed—in a building of its own, well across the IAS campus—the computer had only twenty-three hundred vacuum tubes, considerably fewer than ENIAC’s almost eighteen thousand. It was fully automatic, digital, and general-purpose, but like other programmable computers of its generation, it was built years in advance of programming languages such as Fortran or Pascal. Its commands instead were written directly in machine language, long strings of ones and zeros. An expression such as “A + B,” for instance, might be rendered as something like 01101101 10110110 01110011; a significant program would feature many pages written in such notation. In Bigelow’s words, there were “none of the tricks that we now have. This was a case where Von Neumann was so clever technically that he had no problem with it. And he couldn’t imagine anyone else working with a computer who couldn’t program in machine code.”

How significant was this IAS computer? The science historian Joel Shurkin, who has sought to assess fairly the claims of various inventors as to priority, writes that “Von Neumann’s technical contributions are manifest and beyond controversy. The machine he designed would be faster than anything else.… While all the other computer makers were generally heading in the same direction, Von Neumann’s genius clarified and described the paths better than anyone else in the world could. Moreover, many of the developments in programming and in machine architecture at the institute profoundly influenced future computer development.… While others were using crude digital instructions for their machines, Von Neumann and his team were developing instructions (what scientists call codes) that would last, with modification, through most of the computer age.”

The machine received its baptism with the nation again at war, in Korea, and with the hydrogen bomb now a matter of highest priority. Von Neumann, who had maintained his leadership in nuclear-weapons work, arranged to run a problem dealing with H-bomb physics. It would be the most extensive computation ever carried out. “It was computed in the summer of 1950,” says Bigelow, ”… while the machine had clip leads on it. We had engineers there to keep it running and it ran for sixty days, day and night, with very few errors. It did a nice job.”

The way was open, then, for the computer to sweep all before it. There would be substantial technical advances: programming languages beginning in the mid-1950s, then transistors, integrated circuits, and microprocessors. But these would merely offer better ways to implement the basic concept—the stored-program computer—that Von Neumann had described in his 1945 “First Draft.”

As if computation carried with it some dreadful incubus, a number of its pioneers would die amid tragedy. Alan Turing was the first, in 1954, at age forty-one. Convicted in England of soliciting sexual favors from a teen-age boy, he was given a choice of prison or hormone treatments. He chose the hormones and soon found his breasts growing. Driven to despair, he made up a batch of cyanide in a home laboratory and died an apparent suicide.

For Von Neumann it was even worse. In the summer of 1955 he was diagnosed with bone cancer, which soon brought on excruciating pain. In the words of his friend Edward Teller, “I think that Von Neumann suffered more when his mind would no longer function than I have ever seen any human being suffer.” Toward the end there was panic and uncontrolled terror. Early in 1957 he too was gone.

For Kurt Goedel it was his own per- sonal demons that would drive him to death. In an epic escape from Nazioccupied Austria, he and his wife had crossed the Soviet Union and then the Pacific to reach the United States. From 1940 to 1976 he was himself a member of the IAS. The author Edward Régis describes him, in his last years, as “a cadaverous old man shuffling past alone, dressed in his black coat and winter hat.” After his wife underwent surgery and was placed in a nursing home, in 1977, Goedel refused to take any food. He starved himself to death. When he died early the next year, the death certificate of this great logician stated that the cause was “malnutrition and inanition caused by personality disturbance.”

John Mauchly’s later years were racked by deep bitterness. He was bitter at Von Neumann for not giving him credit as a co-inventor of the storedprogram computer and at Goldstine for being one of Von Neumann’s most effective supporters. Mauchly, along with Eckert, had struggled through lean years but then won success by building UNIVAC, the first commercially successful computer. The firm of Remington Rand brought them in—and eventually the roof fell in. In a major lawsuit their ENIAC patent was invalidated, with the judge ruling that they were not even the true inventors of the electronic computer. “Lawyers keep making money,” said Mauchly toward the end. “We’ve got down to the point where maybe we can buy some hot dogs with our Social Security.” In 1980 he died from a disfiguring genetic disease.

Nor were the pioneering institutions spared, though for them the incubus brought ill-considered abandonment of the computer field rather than unpleasant death. The University of Pennsylvania never recovered from the effects of Irven Travis’s decisions, which cost that school its best computer people. The computer group did manage to build EDVAC, or at least enough of it to satisfy the Army, but it amounted to a last try by this group and led to no new projects. And after Von Neumann died, the IAS abandoned computer science altogether, shipping his computer to the Smithsonian.

The room where it was built, writes Régis, “is not treated as a historical site. No plaque or bust commemorates the birth of the stored program computer within. The room, at the end of a dark and lonely hallway, today houses the Institute’s stationery supplies, and boxes of file folders, pads of paper, and inter-departmental mail envelopes reach almost to the ceiling.”

Yet elsewhere at the IAS, and around the world, are today’s computers, which still follow the directions Von Neumann set forth in his “First Draft” and subsequent writings and that he demonstrated in his project at the IAS. These computers, rather than plaques or busts cast in bronze, are among the true monuments to the cheerful and highly creative man who was John Von Neumann.