Plans and innovation

I am a product of the western world. There are a number of assumptions which go with growing up in such an environment. One of those assumptions is that of progress. We are raised to believe there is some sort of directionality in human development, moving toward some end which is more desirable than the present. Recent history by in large reinforces such a belief system, although there and many people who contest the assumption that what we have experienced represents an improvement over how people lived in the past. I for one think they are crazy and would not for a minute want to roll back time to a point where most children died in childhood and people eked out a day to day existence.

Given our circumstances as people have improved immensely over the past 300 years (see Steven Landsburg - http://online.wsj.com/article/SB118134633403829656.html#articleTabs%3Darticle), it poses a fundamental question. What portion of that improvement was the consequence of specific and intentional human plans and what part was due to unintended consequences of activities committed to for other completely unrelated reasons. For example, did the industrial revolution develop because of a strategic plan put forth by English merchants? Did the German pharmaceutical industry develop because of some master plan devised by the German chemical industry? Obviously the answer to these question is no. Whether it be technological, legal, or social innovations which made quantum leaps possible, the really big ones happened more because the random juxtaposition of events rather than anything planned.

There is no question that certain outcomes clearly benefit from having a well defined plan and defined end points. However, if the game changing breakthroughs are almost always unplanned and linked more to serendipity than planning, what are the ideal rules to implement that fosters both prudent planning and flexibility sufficient to permit disruptive innovation?

I think the key factor is the nature of the challenge one is approaching. There are really three types of problems which we can address. There are simple problems or tasks where the outcomes are clearly definable and the resources and expertise needed to solve them are readily and widely available. An example of this might be the building of a house. It might be expensive and take many months but building a house is a task which has been done literally millions of times.

There are complex and difficult tasks which require coordination of many people and resources over an extended length of time. Some of the challenges may not be fully defined at the time the task is taken on. However, the full scope of the problem can ultimately be defined and resources needed to address the problem identified or created. An example of this was sending a man to the moon. There were initially a number of initially undefined elements but in the end, it was a complex but definable and solvable problem, based upon Newtonian physics, 20th century material science, and for the most part definable variables.

Finally, there are wicked problems (http://cognexus.org/id42.htm). These are problems which we cannot even come close to defining all the variables where the only certainty is the presence of unknown unknowns. Approaching one element of a wicked problem will virtually always have unintended consequences.

Improving the human condition in the long run is a wicked problem. Any intervention is likely to have both planned desirable outcomes as well as unintended undesirable ones. How do we continue to act and not be paralyzed with the fear that our actions will bring disastrous and unintended consequences?

I believe the key to success (as measured continued innovation and progress) is to continue to plan on a relatively small and local scale and to hedge our bets. Provide incentives for people and groups to plan for and gain from small incremental improvements. History would suggest that small wins are like lottery tickets. Acquire enough of them and you will get a game changer. Plan to do too much and attempts to control too much over a time frame beyond which you cannot reliably predict outcomes will generally result only in unintended consequences.