Nowadays, goes the theory, innovation is supposed to be done constantly, by everyone in the company, improving everything the company is about — and new Web-based tools are here to help it happen. Is the theory right? Or do the experiences of companies reveal something different?

Historically, most managers equated innovation primarily with the development of new products and new technologies. But increasingly, innovation is seen as applying to the development of new service offerings, business models, pricing plans and routes to market, as well as new management practices. There is now a greater recognition that novel ideas can transform any part of the value chain — and that products and services represent just the tip of the innovation iceberg.1

This shift of focus has implications for who “owns” innovation. It used to be the preserve of a select band of employees — be they designers, engineers or scientists — whose responsibility it was to generate and pursue new ideas, often in a separate location. But increasingly, innovation has come to be seen as the responsibility of the entire organization. For many large companies, in fact, the new imperative is to view innovation as an “all the time, everywhere” capability that harnesses the skills and imagination of employees at all levels.2

Making innovation everyone’s job is intuitively appealing but very hard to achieve. Many companies have put in place suggestions, schemes, ideation programs, venturing units and online forums. (See “A Glossary of Established Drivers of Innovation.”) However, the success rate of such approaches is mixed. Employees face capacity, time and motivation issues around their participation. There is often a lack of follow-through in well-intentioned schemes. And there is typically some level of disconnect between the priorities of those at the top and the efforts of those lower down in the organization.

Moreover, Web-based tools for capturing and developing ideas have not yet delivered on their promise: A recent McKinsey survey revealed that the number of respondents who are satisfied overall with the Web 2.0 tools (21%) is slightly outweighed by the number who voice clear dissatisfaction (22%).3

To understand these challenges, and to identify the innovation practices that work, we spent three years studying the process of innovation in 13 global companies. (See “About the Research.”) All of these companies embarked on often-lengthy journeys aimed at making themselves more consistently and sustainably innovative. All sought to engage their employees in the process, and all made use of online tools to facilitate and improve the quality and quantity of ideas. Our research allowed us to confirm many of the standard arguments for how to encourage innovation in large organizations, but we also uncovered some surprising findings. (See “Questions That Work — and Don’t — in Online Innovation Forums” for a summary.) In this article we focus on the key insights that emerged from our research, organized around five persistent “myths” that continue to haunt the innovation efforts of many companies.

Myth # 1. The Eureka Moment

For many people, it is still the sudden flash of insight — think Archimedes in his bath or Newton below the apple tree — that defines the process of innovation. According to this view, companies need to hire a bunch of insightful and contrarian thinkers, and provide them with a fertile environment, and lots of time and space, to come up with bright ideas.

Alas, the truth is far more prosaic. It is often said that innovation is 5% inspiration and 95% perspiration, and our research bears this out. If you think of innovation as a chain of linked activities — from generating new ideas through to commercializing them successfully — it is the latter stages of the process where ideas are being worked up and developed in detail that are the most time consuming.4 Moreover, it is also the latter stages where problems occur. We recently conducted a survey in 123 companies, asking managers to evaluate how effective they were at each stage in the innovation value chain. On average, they indicated that they were relatively good at generating new ideas (either from inside or outside the boundaries of the company), but their performance dropped for every successive stage of the chain. (See “Which Parts of the Innovation Value Chain Are Companies Good At?”) We are not suggesting that generating ideas is unimportant. But that is not where most companies struggle. Most companies are sufficiently good at generating ideas; the “bottleneck” in the innovation process actually occurs a lot further down the pipeline.

The eureka myth helps explains why so many companies are drawn to big brainstorming events, with names such as ideation workshops and innovation jams.5 In the course of our research we saw many different types of brainstorming events, and indeed we helped several of the sample companies to put them on. Such events are always valuable: They help to focus the efforts of a large number of people, they generate excitement and interest and they generate some useful ideas.

Myth # 2. Build It and They Will Com

The emergence of second-generation Internet technologies (“Web 2.0”) has had a dramatic impact on how we share, aggregate and interpret information. The proliferation and growth of online communities such as Facebook and LinkedIn seduce us into assuming that these new means of social interaction will also transform the way we get things done at work.

But for every online community that succeeds, many others fail. Some make a good start but then enthusiasm wanes. For example, MyFootballClub is a U.K.-based website whose 30,000 members bought a soccer club, Ebbsfleet United, in 2007. However, by 2010 its paying membership had dwindled to just 800 people, leading to severe financial difficulties for Ebbsfleet United. Other online community initiatives fail to live up to their founders’ hopes. For example, during the transition period before he came into office, President Obama endorsed the idea of an online “Citizen’s Briefing Book” for people to submit ideas to him. Some 44,000 proposals and 1.4 million votes were received, but as theInternational Herald Tribune reported, “the results were quietly published, but they were embarrassing.”6 The most popular ideas — in the middle of an economic meltdown — included legalizing marijuana and online poker, and revoking the Church of Scientology’s tax-exempt status.

How does this affect the process of innovation? Unsurprisingly, all the companies we studied had figured out that the tools of Web 2.0 could potentially be very valuable in helping large numbers of people get involved in an innovation process. Most had built some sort of online forum in which employees could post their ideas, comment and build on the ideas of others and evaluate proposals. For example, IBM used space on its corporate Intranet to launch a 72-hour Innovation Jam in 2006, the purpose being to get IBM employees, clients and partners involved in an online debate about new business opportunities. The Innovation Jam attracted 57,000 visitors and 30,000 posts. A rather different example is Royal Bank of Scotland’s development of a virtual innovation center in Second Life, which allowed the bank to prototype potential new banking environments and get direct and rapid feedback from employees around the world.

In these and other cases, the implicit logic was: Build it, and they will come. Both IBM and RBS had considerable success in attracting interest, but the overall story was much more mixed. Some online forums really helped to galvanize their company’s innovation efforts. Others ended up underused and unloved.

What are the biggest problems with developing online innovation forums? The first is that the forum doesn’t take off. It’s usually quite straightforward to get people to check out a new site once or twice, but they need a reason to keep coming back. As MyFootballClub found, the risk is that the novelty of an innovation forum will wear out pretty quickly and participation will dwindle. A manager at Roche Diagnostics observed: “Our hope that our internal technology-oriented people would gravitate to using this type of tool was completely unfounded. We really had to push people (via an electronic marketing campaign) to involve them in suggesting solutions to the six problems we identified.” Equally, managers at Mars and UBS found their innovation efforts stalling after promising starts. One said: “We probably underestimated the communications needed. We were good up-front, but learned that continuous communications is vital. We had to counter some skepticism, to create the belief that something would happen.”

The second risk is that, like Obama’s Citizen’s Briefing Book, the ideas that get posted are off-topic, half-baked or irrelevant. All the managers we spoke to acknowledged that they had to work hard to “separate the wheat from the chaff.” Many of the ideas put forward were parochial or ill-informed, and few people took the trouble to build on the ideas of others. The notion that the good ideas would be picked up by others and rise to the top rarely worked out.

So what should you do to avoid these problems? The most important point is to understand the types of interaction that occur in online forums, so that you use them in the right way. If you are looking for creative, never-heard-before ideas, and if you want people to take responsibility for building on one another’s ideas, then a face-to-face workshop is your best bet. But if you are looking for a specific answer to a question, or if you want to generate a wide variety of views about some existing ideas, then an online forum can be highly efficient. (See “Questions That Work — and Don’t — in Online Innovation Forums” for examples.)

Takeaway: Online forums are not a panacea for distributed innovation. Online forums are good for capturing and filtering large numbers of existing ideas; in-person forums are good for generating and building on new ideas. Smart companies are selective in their use of online forums for innovation.

Myth # 3. Open Innovation Is the Future

Any discussion of innovation in large companies sooner or later turns to the issue of “open” innovation — the idea that companies should look for ways of tapping into and harnessing the ideas that lie beyond their formal boundaries. Many companies are now embracing open innovation in its many guises. For example, the Danish toymaker LEGO has been leveraging customer ideas as a source of innovation for years, and some new products are even labeled “created by LEGO fans.” And one of P&G’s first experiments with online advertising invited people to make spoof movies of P&G’s “Talking Stain” TV ad and post them on YouTube — resulting in over 200 submissions, some of which proved good enough to air on TV.

Our research confirmed that most large companies believe a more open approach to innovation is necessary, but it also underlined that there is no free lunch on offer. The benefits of open innovation, in terms of providing a company with access to a vastly greater pool of ideas, are obvious. But the costs are also considerable, including practical challenges in resolving intellectual property ownership issues, lack of trust on both sides of the fence and the operational costs involved in building an open innovation capability. Open innovation is not the future, but it is certainly part of the future, and the smart approach is to use the tools of open innovation selectively.

Roche Diagnostics was a company that got a lot of value out of open innovation. In 2009 it put in place an experimental initiative to overcome specific technological problems that were preventing certain R&D programs from moving forward. The company identified six technology challenges that needed solving, and it opened the challenges up to the internal R&D community and to the external technology community through Innocentive and UTEK (now Innovaro), two well-known technology marketplaces. The manager in charge of the initiative described the outcome thus:

Internally, the number of responses to these six challenges was very low. But one very thoughtful response to one of the challenges was brilliant, and paid for the entire experiment. Externally, we used Innocentive and UTEK, and both had a far higher response rate than our internal experiment — more than 10 times the volume of responses, in fact. We offered a $1,500 reward, so this could have been an influencing factor. We received one novel solution, which really made the entire experiment worthwhile, but more than that was our very positive experience of involving external collaborators.

Roche’s experience was the closest thing we saw to a proper experiment that compared the merits of tapping into internal and external communities — and it really highlighted the value of tapping into the external group. But note that the potential respondents were being asked a very narrow, technology-specific question. Clearly, the external community would have been far less useful for tackling company-specific or situation-specific problems.

What are the downsides or limitations of open innovation? One set of concerns relates to how you handle intellectual property issues. At the time of writing, Roche Diagnostics was still working through the details of the licensing agreement with the person who solved its technological problem, and the transaction and licensing costs were far from trivial. A related issue is that without the strong IP protection that a market-maker like Innocentive provides, external parties are careful with what they will share. IBM discovered this in its Innovation Jam. As one manager recalled, “This Jam was established as an open forum, so anyone can take these ideas and use them. So we felt we were taking a few risks doing this, and perhaps it meant that our clients were quieter in the discussions than we would have liked. But it was important to make this open in every sense of the word.”

A second set of concerns was around how the companies we studied actually used the insights provided by external sources. One European telecom company had a “scouting” unit in Silicon Valley to keep an eye on exciting new startups and emerging technologies, but the scouting team discovered that the only technologies the folks back in Europe were interested in were those that would help them accelerate their current development road map. The really radical ideas, the ones that the scouting unit was putatively looking for, were simply too dissonant for the European development teams to get their heads around.

A final concern is simply the time it takes to do open innovation properly. Companies such as Procter & Gamble, Intel and LEGO have put an enormous amount of investment into building their own external networks, and they are beginning to see a return, but you shouldn’t underestimate the time and effort involved.

Takeaway: External innovation forums have access to a broad range of expertise that makes them effective for solving narrow technological problems; internal innovation forums have less breadth but more understanding of context. Smart companies use their external and internal experts for very different types of problems.

Myth # 4. Pay Is Paramount

A dominant concern when organizations set out to grow their innovation capabilities is how to structure rewards for ideas. A common refrain is that innovation involves discretionary effort on top of existing responsibilities, so we have to offer incentives so people to put in that extra effort. The example of the venture capital industry was mentioned as a setting in which people coming up with ideas, and those backing them, all have the opportunity to become rich.

But both academic theory and our discussions with chief innovation officers indicate that this is a red herring.

Let’s briefly look at the theory. People are motivated by many factors, but extrinsic rewards such as money are usually secondary, hygiene-type factors. The more powerful motivators are typically “social” factors, such as the recognition and status that is conferred on those who do well, and “personal” factors, such as the intrinsic pleasure that some work affords. More specifically, there is evidence from psychology research that individuals view the offer of reward for an enjoyable task as an attempt to control their behavior, which hence undermines their intrinsic task interest and creative performance. Parallel research in behavioral economics suggests that intrinsic motivation is especially likely to suffer when the incentives are large.

All of which suggests that you don’t need monetary rewards for innovation. Innovation is intrinsically enjoyable, and it’s easy to recognize and confer status on those who put their discretionary effort into it. Our research interviews provided plentiful evidence that this is the case.

Take the experience of UBS. With considerable upheaval at senior levels of the bank, the innovation movement was very much a grassroots effort — built around “UBS Idea Exchange,” an online tool. The executive in charge of that effort commented: “We found that employees having an opportunity to put forward their ideas brought huge personal rewards. We learned very clearly (through our experiments) that financial rewards would not have made any difference. People reported that recognition of their ideas was a reward in itself. They wanted to be engaged and to participate. We therefore involved people in presenting their ideas to senior management.”

The sentiment was echoed by the head of innovation at Mars Central Europe: “We try to recognize people rather than offer material rewards. We hold a corporate event, biannually, called Make The Difference, where ideas and success stories are celebrated. The Central Europe team is very proud of the fact that we won more awards at this event last year than any other region.”

Takeaway: Rewarding people for their innovation efforts misses the point. The process of innovating — of taking the initiative to come up with new solutions — is its own reward. Smart companies emphasize the social and personal drivers of discretionary effort, rather than the material drivers.

Myth # 5. Bottom-Up Innovation Is Best

There is a lot of enthusiasm among those writing about innovation, and among those working in R&D settings, for bottom-up activism or “intrapreneurship.” The reasoning here is straightforward: Top executives are not close enough to the action to be able to come up with or implement new ideas, so they need to push responsibility for innovation down into the organization. “Let 1,000 flowers bloom” has long been the mantra of big successful innovators like 3M, Google and W.L. Gore.

We wanted to believe this, and we sought out companies that had allowed, or even encouraged, bottom-up processes. We wanted to find cases where dramatic changes had emerged through bottom-up initiatives. But we came back emptyhanded.

Don’t misunderstand. There are plenty of examples of successful innovations that started out as below-the-radar initiatives, or as proposals that got rejected by top executives several times. Examples that spring to mind include Ericsson’s mobile handset business, Sony’s PlayStation and HP’s printer business. But, the point is, at some point all these innovation were picked up and then prioritized by top management. Successful innovations, in other words, need both bottom-up and top-down effort, and very often the link is not made.

During the research, we followed several cases of bottom-up innovation in considerable detail: UBS’s Idea Exchange, Best Buy’s resilience initiative and GlaxoSmithKline’s Spark program. These initiatives were neither great successes nor outright failures. They were able to demonstrate all sorts of modest successes, but they didn’t have the impact that their proponents would have liked either.

We discussed this issue in a workshop in late 2008, and the story that emerged was interesting. An executive working for RBS described the tension he had experienced between a top-down and a bottom-up approach. The company had put in place a range of tools: “Some of these are top-down tools that are owned by senior executives; others are bottom-up tools that we put in place to get involvement from large numbers of people. Top-down we have a group innovation board with senior decision makers and then 12 innovation boards. On a bottom-up basis, each division has its own pipeline, and makes the initial seed investment. Then as costs increase, the idea goes to the innovation board, and if it is approved the board will fund a pilot project, which in turn helps the development of the business plan.”

The underlying point, he observed, is that successful innovation requires close attention to both facets: “We’ve learned that you only get the top-down working if you get the bottom-up right too.”

This interplay between direction and empowerment is evident even in a declared bottom-up innovator like Best Buy. The success of the U.S. retailer is strongly tied to the cumulative effect of continuous experimentation and small bets at the level of individual stores.12 Yet top management plays a significant role in channeling the collective creative energy toward desired areas by framing the innovation challenge in terms of finding new and better ways to service customers (dubbed the “customer centric-cycle”) — hence removing the risks of random or ill-focused innovation.

One final aspect of the bottom-up process is how to deal with those whose ideas are turned down. Broad-based innovation actually implies saying no to a lot of people, sometimes repeatedly. How their contributions are acknowledged, the transparency of the decision-making process and how the news is communicated are crucial factors in keeping the ideas coming. Even when their own ideas are rejected, employees also note what happens to the successful ideas of colleagues — and companies should not underestimate the stimulus of seeing front-line innovators sometimes given the opportunity to implement the ideas they generated. Indeed, Whirlpool, an exemplar in democratic innovation, goes one step further: It has established an Innovation E-Space that allows all employees to keep abreast of innovation activities and even to volunteer to work one another’s projects.13 Once again, the interaction between bottom-up and top-down initiatives proves decisive.

Takeaway: Bottom-up innovation efforts benefit from high levels of employee engagement; top-down innovation efforts benefit from direct alignment with the company’s goals. Smart companies use both approaches, and are adept at helping bottom-up innovation projects get the sponsorship they need to survive.

Conclusion

Innovation is the lifeblood of any large organization, and many invest enormous amounts of time and effort in fostering distributed innovation programs. Web 2.0 technologies have made it possible to democratize the process even further, and offer ways of consolidating and evaluating radically new ideas.

But there are no quick fixes, panaceas or one-size-fits-all solutions — not surprisingly, since by definition not everyone can be a successful leader in innovation.

In this article we have taken an experience-led approach. Forget what the theory says: What are the experiences of companies putting these new tools for distributed innovation into practice? And the truth proves sobering. Online tools, open innovation communities and big collaborative forums all have their limitations. None is always right or always wrong. The best approach involves careful judgment and a deep understanding of the particular challenges a company is facing. By thinking through the pros and cons of each element, companies can manage their processes better.