From the Archives: Resilience vs. Anticipation

What makes Silicon Valley special? A view from 1997.

In 1997, a year before the founding of Google, Rich Karlgaard was the editor of Forbes ASAP, a bimonthly technology magazine published by (surprise!) Forbes, and I was one of his regular columnists. For a special issue, he asked me to write a feature taking on a perennial question: Why did Silicon Valley rather than the Boston area become the leading technology region? The result was one of my classics. Thanks to reader Andrew Mackenzie for recently reminding me of it in a Facebook comment. Please let me know what you think, and how you might update it, in the comments.

Everybody has theories about what makes Silicon Valley special, and most of the theories are right: It’s the density, the competition, the constant chatter about business plans over tables at Il Fornaio in Palo Alto. It’s the universities, Stanford and Berkeley, world-class research institutions that nonetheless nurture the practical. It’s the money, the greatest concentration of venture capital the world has ever seen. It’s the diversity, the immigrants from everywhere, the best and most brilliant spilling out of Oracle’s food pavilions to eat burgers, curry, and sushi in the California sun.

The California sun.

Eventually, all the theories wind up there, at the one thing that makes Silicon Valley unlike Boston, or Austin, or Seattle, the one thing they can never hope to copy: It’s the weather.

The weather in the valley is perfect. Not temperate, not tolerable, not good. Perfect. Month after month after month of sunny days, with just enough breeze and humidity to keep your skin happy. Prelapsarian weather, the stuff of utopian dreams.

People from elsewhere think Californians are crazy—or terminally superficial—to go on and on about the weather. That’s because it has to be lived to be believed, or appreciated.

Zero-point-zero inches, read the four-month rain report in the local paper when Bart Kalkstein moved to the valley from Philadelphia in 1995. Before then he’d never understood why his Princeton roommates, Bay Area natives, wouldn’t shut up about the weather. “When they told me it doesn’t rain,” he says, “I had thought it doesn’t rain that often—you probably only get a couple of nights here and there a week. But, literally, it doesn’t rain.” Now product director of Sterling Software's desktop integration division, Kalkstein, 27, grew up in Massachusetts.¹ He never imagined such a thing. “The weather is by far the most important: influence on local culture, he says, the key to its energy and a magnet for youth.

The tawny hills of Silicon Valley, photo by Evan Blaser.

I believe him. In 1986 I came to California from Boston, a city that proclaims itself “the Hub of the universe” yet seems eternally hunkered down, hemmed in by clouds, darkness, and narrow streets. Recalling elementary school history lessons as I trudged to work against a below-zero windchill, I used to wonder how any Pilgrims survived that first winter. I was sick half the time, and sleepy the rest. California made me happy, well, and energetic. I could work twice as hard with half the effort. If you want to know why the valley has supplanted Boston—which also has brains, money, and a proud technological and business heritage—as the Hub of the high'-tech universe, you can’t ignore the weather.

“There’s nothing more depressing than being in East Cambridge on a rainy, cold spring day, looking out at the river and having the wind blowing. It's just generally nasty. It doesn’t inspire you,” recalls Tom Henry, now president and CEO of Quote.com, an Internet provider of financial-market information based in Mountain View, California. “Out here, the sky's crystal clear. It makes a huge difference. No matter how bad your day is, if you look out the window and it's sunny, it's pretty hard to stay down for too long.”²

Silicon Valley’s perfect weather means you don’t need backup plans, just in case it rains. It means you don’t resent spending a beautiful day inside at work, because tomorrow and tomorrow and tomorrow will be just as gorgeous. It means you have more energy, sapped neither by sleep-inducing clouds nor enervating heat and humidity. It means fewer days dragging into the office with a brain dulled by allergies and winter colds. It means you have more life.

But against the beautiful blue skies of the valley sprawl its tawny hills, their curves clearly visible beneath a bare wisp of foliage. In the dry landscape of the West, the earth is not camouflaged by trees and vines and underbrush. In the valley, the ground itself is omnipresent.

And, as everyone knows, it is also unstable.

On October 1, 1987, as I was about to step into my morning shower, my apartment began to shake—just as it had once shaken when the Boston T went by, rising on aboveground tracks to cross the Charles River. But this apartment was in West Los Angeles, where, I realized in an adrenaline spurt, there is no train. The shaking clocked in at a magnitude 5.9.

Two years later, while Bay Area residents eagerly awaited the beginning of a World Series game between the Oakland A's and the San Francisco Giants, the earth beneath them convulsed in a 7.1 quake. The Oakland-San Francisco Bay Bridge buckled, sending a 50-foot top span crashing through the level below. A mile-long, double-deck section of Interstate 880 collapsed, crushing more than 100 vehicles and killing 42 people.

Every year on Yom Kippur, Jews recall the people of an ancient, unstable region of the Middle East, for whom it was prayed that “their houses not become their graves.” The prayer occurs in a relatively obscure part of the service, a section many people skip. But in California, they pay attention. To live in earthquake country is to know, way back in the back of your mind, that your house, your car, your office could—at any moment—become your grave. All your worldly goods could crumble in an instant. That, too, makes Silicon Valley special.

“My wife and I honeymooned in San Francisco,” says Art Hutchinson, a technology-management consultant with Northeast Consulting Resources in Boston. “We'd love to live there—except for earthquakes. We're willing to put up with months and months of regular pain, sort of do our penance in the dark and cold, shoveling snow on April 1 in a freak blizzard. But the risk of one great whomp and you’re flattened on the 880? No. . . . That feels very parallel to the risk profile that people think about in business. If you’re willing to move there, you’ve already accepted a certain subliminal level of risk.”

A predictable natural disaster

Boston’s winter is a natural disaster, but its predictability changes everything.³ As Hutchinson suggests, New Englanders know winter is coming. Bad weather is annoying but easy to plan for: You build snow days into the school year, buy a car with four-wheel drive, get used to scraping ice and shoveling snow. You make sure you have a coat, hat, and gloves. Snow, says Hutchinson, is no big deal: “You just put on boots.” Life has a regular rhythm.

Good weather plus earthquakes creates an utterly different environment. On a day-to-day basis, you can concentrate on your goals, with no need for contingency plans. Your softball game, your picnic, your wedding won’t be rained out. But everything could change in an instant. You can’t anticipate earthquakes, can’t plan for them, can’t even predict when and where they’ll strike. Instead of providing the certainty of seasons, nature promises a future of random shocks. All you can do is develop general coping skills and resources. There is nothing familiar about the aftermath of an earthquake, and no one survives it alone.

In his 1988 book, SEARCHING FOR SAFETY, the late UC-Berkeley political scientist Aaron Wildavsky laid out two alternatives for dealing with risk: anticipation, the static planning that aspires to perfect foresight, and resilience, the dynamic response that relies on having many margins of adjustment:

Anticipation is a mode of control by a central mind; efforts are made to predict and prevent potential dangers before damage is done. Forbidding the sale of certain medical drugs is an anticipatory measure. Resilience is the capacity to cope with unanticipated dangers after they have become manifest, learning to bounce back. An innovative biomedical industry that creates new drugs for new diseases is a resilient device. . . . Anticipation seeks to preserve stability: the less fluctuation, the better. Resilience accommodates variability; one may not do so well in good times but learn to persist in the bad.

Here, then, is the basic difference between the Valley and the Hub: Viewing the world as predictable and itself as the center of the universe, Boston has encouraged strategies of anticipation. People try to imagine everything that might go wrong and fix it in advance. But in Silicon Valley, there are no certainties. The future is open and subject to upheaval. Resilience is the strategy of choice. People do the best they can at the moment, deal with problems as they arise, and develop networks to help them out.

The positive side of anticipation is that it encourages imagination and deep thought, the stuff of intellectual life. And it is good at eliminating known risks. It can build confidence, appropriate in a city whose leading business manages the life savings of America’s middle class. “Just look at the name: Fidelity,” says Hutchinson. “Rock solid, New England.” Building Fidelity Investments meant not just hiring good fund managers but getting customers to trust impersonal computers, phones, and customer-service systems as they’d once trusted the neighborhood banker. Anticipation makes such systems reliable.

But anticipation doesn’t work when the world changes rapidly, and in unexpected ways. It encourages two types of error: hubristic central planning and overcaution. To find the former, there is no better source than old copies of the Harvard Business Review, where industrial policy gurus once plied their trade. “Most experts believe that without deep changes in both industry behavior and government policy, U.S. microelectronics will be reduced to permanent, decisive inferiority within ten years,” wrote Massachusetts Institute of Technology’s Charles H. Ferguson in a famous 1988 article. Who were the experts?

A wide number of university researchers and senior personnel of my acquaintance in the U.S. Defense Department, the CIA, the National Security Agency, the National Science Foundation, and most major U.S. semiconductor, computer, and electronic capital equipment producers. My conclusion, after meetings with groups in the U.S. Defense Science Board, the White House Science Council, and others, is that only economists moved by the invisible hand have failed to apprehend the problem.

Ferguson and his mandarin contacts anticipated disaster and demanded industrial policy because their imaginations failed them. They didn’t envision an industry driven not by DRAM-making but by microprocessors, software, and networks.

To anticipate is to worry: Mandarins worry and propose dramatic plans to fight their fears. Regular businessfolks worry and fail to act. Consider the eternal bug fight. Anticipation says you should try to get all the bugs out of a product before launching it: zero tolerance for error. When Quote.com’s Tom Henry talks to his old colleagues at BayBanks in Boston (now merged to form BankBoston), he can hardly believe what they tell him: “They’re still trying to get out home banking—and this is 15 years later. Holy smokes! In what other business could you wait 15 years to get a service out?” But no one wants to make a mistake.

Kimbo Mundy, now president and CEO of Bidder’s Edge, a Wellesley, Massachusetts, startup developing a Web-based venue to provide information, tools, services, and contacts to the online auction market, spent 12 1/2 years at Interleaf, first in Cambridge, then in Waltham. He is proud of the work he and his colleagues did there, solving difficult programming problems. His intellectual side considers the Silicon Valley approach to software superficial. But, from a practical point of view, it makes sense. “Being a little bit more superficial, slapping something out there just to get it done and be able to make noise about it is actually the way you should develop software,” he says. “The hardest thing is to just get the customers’ attention in the first place. Once you get their attention—as long as you haven’t pissed them off—you can then do release after release and, like Microsoft, by the time you get to release 3, you actually have a product.”

Indeed, trying to identify and anticipate every bug—and to make the programming as elegant as possible—undercut Interleaf’s technical achievement by limiting its market. “We had a product that was the best electronic publishing product that could be had in the mid-’80s,” recalls Mundy.⁴ “It did things that the intellectuals of the time, like Brian Reid who did Scribe [a text editing system] at [Carnegie Mellon University], said couldn’t be done. We were a WYSIWYG system with all sorts of fancy formatting. And we sat on that product for a year cranking bugs out of it. . . . That was right about the time that Frame Technology came out and started to eat our lunch.”

Frame, a San Jose Company now part of Adobe Systems, also had another advantage: It used the Sun Microsystems user interface, while Interleaf insisted on its own. The do-it-all-yourself approach, which makes anticipation and control easier, is the touchstone characteristic of Boston’s high tech community, reflected in proprietary systems and company-man loyalty. Boston traditionally keeps everything within the same box. By contrast, the West Coast industry is built on open systems and professional mobility, both of which enhance resilience.

“The minicomputer generation was all about proprietary franchises, proprietary operating systems and architectures, and trying to corral the customers into believing that they could only do their work with one person's hardware,” notes Tom Henry. Ten years ago, he was working at Prime Computer (now Computervision) in Bedford, Massachusetts, and suddenly realized, “I gotta get out of here, because hardware is going to become a commodity.” He left to sell banking systems for a financial software company in Cambridge. Today, he’s in a Silicon Valley business built on partnerships, specialization, and resilience. An Internet-based service, Quote.com isn’t tied to any particular hardware or network. It integrates financial news and research from a bunch of different partners and delivers it through still others, as well as through its own site. For its part, Quote.com provides one-stop shopping and a software-based editorial filter, organizing and selecting content with the habits of “serious investors” in mind.

“Our vision is you build a fabric of partnerships and it’s a lot more resilient to changes in the business,” says Henry. “On the East Coast, there’s still the perception—the dying perception—that, ‘Well, we can do it all on our own. We’ll just keep adding the people and we’ll develop those core competencies.’” A web of partners provides more “sources of field intelligence” to spot changes in the marketplace. Partners from different industry backgrounds and corporate cultures offer a diversity of perspectives and ideas for responding to new situations. It’s also far less painful to drop a failing partner than deal with a floundering division. For new brands like Quote.com, partnerships also offer a way to build credibility quickly.

When exactly partnerships make for resilient companies is a deep question, one that goes to a very basic puzzle for economists: Why are there firms? Why doesn’t Tom Henry just operate Henry.com and contract for everything? Clearly some integration is useful, and under some circumstances it may even make for corporate resilience. A century ago, meatpacking innovators Swift and Armour used vertical integration to build the resilience to face uncertainties of weather, animal sickness, transportation breakdowns, and other shocks.

There’s no question, however, that over the past few decades Silicon Valley’s open systems have proved more resilient than the old proprietary boxes. Venture capitalist Paul Koontz of Foundation Capital in Menlo Park points to Sun as the model: It started with off-the-shelf components and a public domain operating system, and it developed networking standards that allowed computers from many different manufacturers to swap files. “What a wild concept in the computer business back in the early days,” says Koontz. But, he notes, “it was very comfortable for companies out here to think that way.”

The contrast between Sun and Apollo Computer, its onetime rival in the workstation market, comes up again and again in these discussions. Apollo, which was based in Chelmsford, Massachusetts, built a traditional proprietary system and an equally traditional corporate culture. In a 1987 Wall Street Journal feature, Apollo chairman Thomas Vanderslice dismissed Sun’s playful atmosphere with a contemptuous cliche: “In this country, everything loose rolls to the West Coast.”

While Sun was slashing prices and going after every market that would have it, Apollo took a more cautious approach. “I’d like to consider ourselves good businessmen,” Apollo president Roland Pampel told High Technology Business in 1987. “If we’re going to go into a market, I’d hope that we would know how we're going to support it.” Such careful anticipation had already made Apollo vulnerable to shocks, however: A 1984 slowdown in the semiconductor business, its base market, sent it scrambling to develop new customers—at the very time Sun was making inroads with its cheaper, Unix-based machines. Eventually, Apollo, too, had to cut prices and adopt Unix. But it still wound up in a downward spiral and was eventually acquired by Hewlett-Packard. In a rapidly changing technology environment, Apollo was never able to turn its desired “image of solid, long-term reliability,” as the Journal summarized its 1987 strategy, into something for which customers would pay a premium.

Sun’s initial strategy of off-the-shelf components, by contrast, allowed it to quickly incorporate technical advances, rapidly improving its price/performance ratio in its early days. And, echoing Wildavsky’s notion that resilience “accommodates variability; one may not do so well in good times but learn to persist in the bad,” CEO Scott McNealy emphasizes that the company’s use of open interfaces gives other companies a stake in its success. Sun foregoes a chance to control the whole market—the ultimate goal of a strategy of anticipation—in exchange for spreading the risk and, of course, the reward.

Resilience, however, is less a guarantee of corporate success than it is a way of reducing the risk for individual careers and the regional economy. A strategy of resilience means not that companies won’t fail but that resources—including human resources—are more likely to move to better uses more quickly, with less trauma. Indeed, the willingness to abandon losing projects is fundamental. The idea is to adjust quickly, on a small scale, rather than all at once: to be like grass bending before the wind, then springing back, rather than a solid oak that comes crashing down in a storm. In a resilient economy, employees have choices, and they move around.

Job-hopping is inevitably what Easterners single out as Silicon Valley's most distinctive cultural trait. Employees there, notes Foundation Capital’s Koontz, offer companies “transient loyalty”: “When people are engaged with a company, they are passionately committed to it—as opposed to obligingly committed, as one might have been in a 30-year career at IBM.” Inspired, they put their lives on hold and focus all their energies on their work. “Having said that,” Koontz cautions, “people have demonstrated an ability to magically lose that loyalty on very short notice and go and rev up an equally passionate commitment to somebody else.” For the valley’s mobile professionals, employment is another partnership in a resilient network of relationships. It isn’t a lifetime commitment on either side.

San Francisco’s Marina District after the 1989 Loma Prieta earthquake (U.S. Geological Survey photo)

The valley of perfect skies and unstable ground does not inspire thoughts of permanence. Blending intense ambition and constant upheaval, it has created a carpe diem culture of transitory greatness: You achieve today because everything could change tomorrow. Boston is also a stimulating intellectual environment and still produces a steady stream of startups. But the regional culture of anticipation means technologists and entrepreneurs must struggle between its two extremes: too much control and too little closure. Hutchinson, the Boston technology-management consultant, draws the contrast between the hierarchy of Harvard—“a pecking order with a capital P”—and the openness of high tech ventures, such as Digital Equipment, that sprang from the MIT culture. That culture, he notes, is to say, “Everybody’s interesting. Every thing is interesting. Every possibility is interesting. Let's discuss things. Let's have more conversations. Let’s debate stuff. Because it’s just interesting. Oh, we need to turn out a product. Gosh.” That pure, impractical culture of visionary play may generate wonderful ideas, but someone else must turn them into realities.

Kimbo Mundy of Bidder’s Edge illustrates the difference between the coasts with two technological heroes: Tim Berners-Lee, now at MIT, who invented the underlying structures of the World Wide Web, and Marc Andreessen, now in Silicon Valley, who created the good-enough integration of text and graphics that made it take off. Each migrated to the appropriate cultural milieu. “On the East Coast,” says Mundy, “it’s the building of the thing that’s most important. And on the West Coast, the sharing of it is relatively more important. Getting things out to the light of day seems more important there.”

Once they hit the light, no one can anticipate just where innovations will lead--or whether they will in fact succeed. It is by trusting the search, permitting experiments whose results no one can know, that we allow advances to occur. In a 1979 paper, Wildavsky prefigured his discussion of anticipation and resilience with a meditation on the sources of progress. It depends, he suggested, on spontaneity and serendipity, on discoveries no one can predict or foresee:

Incessant search by many minds...produces more (and more valuable) knowledge than the attempt to program the paths to discovery by a single one....Not only markets rely on spontaneity; science and democracy do as well....Looking back over past performance, adherents of free science, politics, and markets argue that on average their results are better than alternatives, but they cannot say what these will be....The strength of spontaneity, its ability to seek out serendipity, is also its shortcoming--exactly what it will do, as well as precisely how it will do it, cannot be specified in advance.

Nowadays it seems that every place wants to be like Silicon Valley—to discover its secrets and copy them. Here, then, is a secret that can be copied, even in places with lousy weather and stable ground: Don’t ask for answers in advance. Don’t try to create a life without surprises. Trust serendipity.

1. He returned to Massachusetts to go to Harvard Business School and has been there ever since, according to his LinkedIn profile.

2. He also went back to Massachusetts.

3. “We have a natural disaster every year in Chicago. It’s called winter,” the great economic historian Joel Mokyr said to me when I was researching this story. It’s one of my favorite quotes but it got edited out of the final version because it was too hard to set up. Joel made the point that the sheer expense of coping with winter is enormous but never counted in the cost of natural disasters because winter is predictable.

4. He’s now in Albuquerque. As a reporter for Inc. magazine, I saw an early demonstration of Interleaf’s desktop publishing system. It was amazing, definitely like something from the future.