Catching up with an old friend in the last few weeks has brought this home to me. He is a highly capable and dedicated individual, but he describes the last year as “difficult”. His organization has recently spun off from its parent in a management buyout, so everyone is working hard to make the numbers and repay the debt. He is a devoted father who dotes on his two children. He told me that his son, who must now be eleven or twelve, is in remission. My friend says he tried to carry on his work from the hospital for six months. To add to his trials, a tendon snapped in his leg. He does not know how. When they operated, they found two tendons had broken. It took five hours to repair the damage

This all sounded horribly familiar. The body has ways to make us slow down whether we plan to or not. I move in a sailing community and health scares are a common denominator of many of the people here. Most were reasonably successful in earlier careers or they could not afford this life. Stress-related illness forced many to rethink their lifestyle: migraines, chronic back pain (my case), high blood pressure, heart attacks, insomnia, ulcers, arthritis, and cancer.

Working harder is simply not an option, despite shrinking markets and increasing competition, for that way lies burnout. Too many of us work too hard already.

Nevertheless, working smarter is not easy, either - especially under stress. It is OK for Daniel Pink to say that knowledge workers are toast (automation or outsourcing to Asia will take these jobs away), and that the future belongs to the Creative. However, how do knowledge workers become creative? Instead, we continue to scurry around faster and faster, busy idiots in fear of the unknown.

Working smarter is not easy for individuals, and certainly not easy for organizations. Nevertheless, there is hope on the horizon.

The organizational challenges were the subject of Dick Foster’s Innovation: the Attacker’s Advantage (1986). He had just completed a successful decade as a management consultant, tapping electromechanical manufacturers on the shoulder to warn them about the impact of electronics. Almost all said they already knew about this, had hired bright electronic engineers, and had introduced an electronic product in their field (cash registers, typewriters, telephone exchanges and so on). However, if pushed, they would add that the electronic product cost more and seemed to serve a different market. Furthermore, they would ask, what else could we do with our press shop and assembly workers than to continue to make the electromechanical products for which there is still a demand?

These managers were, of course, mostly electromechanical engineers invested in a technology leapfrogged by electronics. Economists might well have told them that sunk costs are sunk costs… but it is very hard to see this. Especially as accounting enshrines sunk cost in capital employed, and CEOs expect a return on that capital.

This gives the attacker the advantage. When, inexorably, the cost of electronic products declined making the traditional electromechanical product uneconomic, the incumbent manufacturers failed to survive. Even in the narrower field of semi-conductors, valve manufacturers did not become transistor manufacturers, transistor manufacturers did not become integrated circuit manufacturers, and integrated circuit manufacturers did not become the leaders in microprocessors.

It is a great story, and in 1987, Clayton Christensen came out with another book with a similar title and a few more twists: The Innovator’s Dilemma. Nothing fails like success, he told us. The success an organization achieves from its original innovations makes it difficult for it to be innovative and creative in the future.

The dilemma arises when the organization has lost its stomach for failure. Innovation requires failure. If you do not fail in the process, are you taking enough risks, are you really being creative and innovative? What happens to the individual that truly believes in his or her "out of the box" idea and continues to push it forward when it does not fit within the historical norms of the organization?

These books and such questions were the grandparents of the Yala.

The message of The Innovator's Dilemma resonated strongly with one of the world’s largest oil companies whose CEO saw himself in the mould of GE’s Jack Welch – ruthlessly demanding performance and swiftly punishing failure (two strikes and you are out). Quite the opposite to the apocryphal story of the employee who comes to their boss, after making a million dollar blunder, expecting to be fired, only be to be told “I would be crazy to get rid of you now, I have just spent a million dollars educating you”.

This international oil company had recently grown through mergers with two other oil companies, and wanted to inject fresh ideas into the work of its refineries. At the heart of such operations lies Linear Programming. This is a mathematical technique. It seeks to optimize the economics of an oil refinery by balancing the output of the ethylene crackers (those tall thin reactor vessels that tower over refineries like spaceships on the launch pad) and the output of any downstream chemical plant, against the supplies of crude oil and the demands for refined products and chemicals. Linear Programming had not changed substantially for a number of years. Every company approaches it slightly differently and each refinery believes they get it more or less right.

Aware of the dilemma for innovators in its own results-driven culture, the oil company made the remarkable decision to outsource this key activity in the hope that outsiders would bring a new focus and, one day perhaps, pave the way to balancing supply and demand not for each refinery separately but for the network of refineries as a whole. As the deal came into effect the single relationship between the outsourcing company and the oil company’s procurement team had to roll out to two dozen refineries across the world, each an interface where the new arrangement must work.

Though all such deals start with a fanfare of trumpets, many end in disarray and recrimination. Literature on business partnerships still focuses on well-known solutions like cutting the “right” deal. Therefore, lawyers (who are the ones who usually have to clean up the mess when joint ventures and other complex relationships fail) continue to dream of “a fence at the top of the cliff rather than an ambulance at the bottom”. Nevertheless, they know that an elegant contract can no more ensure a successful business relationship than a prenuptial agreement can guarantee a happy marriage.

I had been working on a similar issue in my organization – how to build Quality into our value-delivery process – and had championed a “Build-Quality-In Toolkit”. I agreed with the oil company that if they would supply the IT for an extranet to build quality into their relationship management process, then I would supply the intellectual capital. The tools that proved most valuable back then became the seeds of today’s Yala. Unknowingly, I had done precisely what Eric von Hippel (the high priest of customer-driven innovation) recommends: provided a toolkit for user innovation (see Chapter 11 of his 2005 book Democratizing Innovation, which you can download in full for free at the end of this article).

This then is the hope for knowledge workers in this challenging world. Though we cannot individually expect to metamorphose into a Creative, we can do much to become collectively smarter. At the heart of this lies the Discourse Management Process, using the architecture and discipline of the Yala to improve the quality of dialogue wherever people from different organizations, or different parts of the same organization, have to work together to get things done.

Smart collaboration via the web is a relatively new phenomenon, post dating Foster and Christensen, which goes a long way to addressing their questions. Nevertheless, people still ask me what could possibly motivate this type of collaboration, remembering perhaps the disappointing early days of CSCW (Computer Supported Cooperative Working). The short answer –as Dan Pink notes – is intrinsic more than extrinsic reward. A longer answer is supplied by Tom Malone in his paper Harnessing Crowds: Mapping the Genome of Collective Intelligence, also downloadable at the bottom of this page.

Once again, the bottom line is work smarter, not harder. So get smart, get a Yala.