Chapter
4
The
New Science of Complexity
Birth of a New Field: Complexity is a relatively new science that is inspiring the imaginations of people from a myriad of disciplines, especially those from the various biological-sciences. Researchers from several different fields are coming to realize that linearization and reductionism are insufficient methodologies for explaining the phenomena witnessed in their studies. The behavior of a single water molecule can no more explain fluid mechanics than can an isolated neural activity describe the behavior of the brain. Complex asymmetric systems that are far from equilibrium tend to exhibit bizarre, chaotic behavior in a strangely ordered way. They self-organize and evolve characteristics that are not in themselves uniquely derivable. When systems self-organize and become more complex, they become chaotic yet ordered, and unexpected reactions – such as “emergent” behavior – occur at the instabilities. Complexity is what makes biology work.
But what is complexity
exactly, and why did it seem to suddenly come about? Our fundamental model of the macroscopic
world centers around the precepts of “classical” physics, or the Newtonian
paradigm – named for the English scientist Sir Isaac Newton. One can think of the Newtonian paradigm as
the entire attitude and approach to the physical world that arose after
But there was more. Seemingly new characteristics of systems recognized as complex were beginning to be noticed. The appearance of these attributes is referred to as emergence. When a system can not be predicted or explained from its preceding conditions, it is said to be emergent. Emergence is the condition of a system or entity whereby it can not be explained or defined in terms of the properties or characteristics of its parts, nor can it be reduced to these properties. The origin of life and the rise of sentience in higher-order species are two examples. It might be said that the origination of the concept of complexity was a necessary condition or logical result of the dominant formalism of the Newtonian paradigm reaching its limit. Something more was needed to explain the numerous series of events that seemed to extend far beyond any classical description offered by Newtonian physics. The Newtonian paradigm models the world as simple systems and mechanisms. Complex systems fall outside of that. Hence the field of complexity was born.
Limits
of Reductionism. Newtonian physics
is clean, predominantly linear, and predictable. And its beauty resides in its elegance. For
our everyday macrocosm, it seems to work – and work very well! In fact, Newtonian physics is the very
backbone of modern science and technology.
But Newtonian physics subscribes to the premise that all of what is
known of physical reality can be broken down into its most fundamental
components, or particles, and the forces and laws that govern and act among
them. This viewpoint is a reductionist philosophy. The methodology of modern science centers
very heavily around reductionism. Reductionism is defined in the Microsoft
Encarta College Dictionary [2] as
follows: “1. The analysis of something
into simpler elements or organized systems, especially with a view to
explaining or understanding it; 2. The
oversimplifying of something complex, or the misguided belief that everything
can be explained in simple terms.”
It is fair to say
that the realm of the gods is pushed back by knowledge. When the first curious early hominid looked
out over the vast savanna or peered through the trees of a primordial forest,
he was filled with wonder. Where does
the Sun go at night? Where do the winds
come from? What are the clouds made
of? The human mind needed to know. But more than that, it needed an acceptable
explanation compatible with the tribe’s perception of its local world. Natural phenomena came to be the possessions
and behaviors of a legion of gods and goddesses. When the gods were unhappy, it stormed. When they were content, their sighs were the
breezes. When one displeased the gods,
he would be taken ill. Where there was a
foreboding portent or an apprehension of misfortune, they would send comets
across the night sky. The responsibility
for the control of Nature fell to the gods.
As a result, the gods have domi
But as the human race became more civilized and began to live together in larger groups, needs arose for a more ordered control of the environment. The construction of dwellings, the development of agriculture, and the herding of animals all required humans to now learn the behaviors of specific aspects of Nature such that they might select the correct building materials, know the growing seasons, and breed the animals for the desired characteristics. Knowledge began to displace the gods’ domain. But a primordial drive keyed with our instinct for survival is a fear of the unknown, and this fear has driven mankind to find a means of controlling the unpredictable. The formation of cities introduced specialized tasking as a more efficient means of providing for the common welfare and protection of the community as a whole. This specialization was sufficiently effective in providing for the basic needs that it enabled the luxury of allowing some of the tribe’s individuals to spend time thinking, to direct intellectual energy towards the discovery of the structure and patterns of the environment – the birth of what we call science.
Science
looks into the dynamism and disorder of natural phenomena and tries to extract
a consistent pattern with order and meaning that will explain these phenomena
and coordi
What, then, might all of this have to do with reductionism? To a very large extent, reductionism is a dominant precept of the scientific method. One observes and records a specific phenomenon. One then either analyzes its peculiarities based on previous theory or synthesizes its cause and effects from apparent facts serving as evidence. Either way, a hypothesis is formed from which a theory is constructed. The characteristics are further unified and simplified – and typically “broken down” to its simpler components to understand the underlying physics. Reductionism can be thought of as a process or methodology of attempting to understand a complicated entity, system, or phenomenon by breaking it down into its most basic constituents and studying these components. It is predicated on the premise that by understanding the behavior of the individual components, one can come to understand the whole system. An automobile is comprised of a chassis, running gear, and an engine. The engine is made up of a crankshaft, valves, pistons, and so forth. By reducing each component to its most fundamental processes and understanding how each functions, one can develop a very good comprehension of how an automobile works. This technique is very powerful and has been very successful as a scientific methodology. The very foundation of our fundamental knowledge of classical physics today is based upon the efforts and insights of those tenacious explorers who patiently broke the structure of matter down to its elemental essence. Reductionism has allowed physicists to come to understand the nature of matter on the most minute scale. The benefits derived from the understanding of the constituents of matter enabled by the reductionist approach has given mankind unimagined control over his environment. The success of this process has imbued a paradigm in the minds of many scientists that reductionism is not only the best way to understand Nature, but the only way.
Reductionism can trace its roots from Occam’s Razor, named for the English philosopher and theologian William of Occam (1284 – 1347), whose works helped the transition from medieval to modern scientific thought by basing scientific knowledge on experience and self-evident truths and the logical outcome from either. He emphasized never multiplying entities beyond what is necessary, but instead state the problem in its most basic and simplest terms. The simplest theory to fit all the facts should be the one selected.
A corollary to reductionism is determinism, which is based on the premise that if one is given enough information (data), then everything is predictable. And one obtains such information about a complex system by disassembling it and reducing it to its simplest parts. Classical physics is not only reductionist, but it is also rigidly deterministic. Every cause has an effect, and every effect can be traced to a cause. Or so it would seem. At the beginning of the Twentieth Century classical physics with its reductionist and deterministic arrogance was found woefully deficient in the face of the newly discovered quantum physics – the first significant chink in the impervious armor of reductionism.
The success of reductionism is undeniable. But recent scientific investigations are more frequently giving rise to problems when reductionism is followed as the only process. [3] One example of this phenomenon is the practice of medicine. The reductionist approach to medicine attacks each new symptom with a new medication, wholly ignoring the side effects on the patient, while a wholistic approach that considers a person’s total being and lifestyle is proving more successful. In engineering processes, such as the construction of a coal-fired powerplant, the conventional reductionist approach focuses solely on the immediate and well-understood result of producing electricity, while ignoring the catastrophic effect on the environment. The total system for a powerplant should include the fuel processing (i.e., mining) through the treatment of the powerplant effluents. In “Figments of Reality” Ian Stewart and Jack Cohen submit that all interesting systems – from DNA to mammals to cultures – are much too complex to be understood from a reductionist approach. [4] The most simple of rules in the most simple of mathematical games lead to vast permutations of possibilities with intricately complex patterns and structures that one could never predict from the initial conditions. The simple rule for the movement of each piece in a chess game, for example, leads to virtually an infinite combination of possibilities on a board of but 64 squares. The complex organization and group behavior demonstrated by an ant colony could never be predicted from the individual ant and the very simple chemical rules it follows. It can only be appreciated wholistically. But what is this concept of wholism, and why may it introduce information not available through reductionism?
Wholism.
The concept of wholism is an
implicit quality of complexity. Because
wholistic phenomena borne of complexity is an implicit part of this thesis,
this topic warrants a short discussion.
The “whole” of something can sometimes be thought of as an essentially
structureless unity comprising an observer’s aggregate, described in terms of
its organized parts. As we saw in the
discussion on reductionism, often merely understanding the properties of the
individual parts is sufficient for understanding the whole. In other more simple cases the properties of
its parts can be ignored without appreciable loss of its understanding. But when these parts are complex and
interrelated, they merge to develop other properties that domi
Wholism could be interpreted in one context as a methodological assertion, implying that the best way to study complex behavior is to treat the system as a whole, as opposed to the reductionist approach of merely analyzing the structure and behavior of its parts. The reductionist precept was discussed in the previous section. Another interpretation is to take wholism as a metaphysical thesis suggesting the existence of some wholes whose characteristics simply are not determined by the nature of their parts. While methodological wholism balances against reductionism, especially in the sciences, metaphysical wholism more closely conjures the notion of pure nonseparability. A nonseparable wholistic phenomenon can be thought of as that whose whole properties can not be determined by the properties of its parts. But metaphysical wholism implies such concepts as some objects not entirely being composed of basic physical parts (onthological wholism), some objects having properties not determined by the physical properties of their basic physical parts (property wholism), and objects obeying laws not determined by fundamental physical laws governing their basic parts (nomological wholism). [5] Although examples of such characteristics may exist in quantum physics (photon entanglement, for example, which is a non-factorable sum of product states), clear examples of pure nonseparability are not apparent in classical physics. Metaphysical wholism is mentioned here mainly for completeness. Methodological wholism captures more of the essence of what is implicated in the debates about wholism in the “complex” sciences, such as biology and sociology, and it is this perspective of wholism that is implied in this thesis – but with the caveat that the emergent phenomena evident through the wholistic presence can be lost once the system is disassembled.
When one speaks of wholism, one speaks of a process of directing attention to the “whole” of something and to its characteristics as a whole, all without a specific consideration of the behavior of its individual parts. And there are many aspects of wholism. For example, one could be wholistic about a theory confirmation to the extent of how empirical claims must force the experience of a system as an aggregate while not being wholistic at all about the belief in that theory. As a practical example of what this statement implies, Sherlock Holmes was famous for asserting that when all of the possible solutions to a problem have been expended, then the only remaining conclusion is the impossible one. The experience of the aggregate system forces that conclusion upon the observer, but one may be reluctant to believe it. The system of expended solutions to the problem is one wholistic class, and the state of being of disbelief is another. But what the experience is likely dictating, however, is that one has not truly perceived the problem in its correct entirety, thus implying yet another wholistic state. The latter half of the Twentieth Century saw much wholistic philosophy applied to disciplines as diverse as the biological and social sciences or the concepts of mind and language. A single cell of the body, albeit comprised of various subsystems of genes, DNA, and other molecules, nonetheless is identified uniquely as a cell possessing characteristics that exceed the sum of its parts. As one speaks or reads a sentence, he understands the meaning of that sentence as it relates to other sentences in the language in which the sentence is expressed. Language itself is wholistic in that understanding emerges from an apparent set of nonsense syllables.
But
as one reviews the principles of wholistic systems, one might be intrigued to
learn how little is understood about the true universalistic nature implied by
wholism. Nature itself is a wholistic
structure comprised of a hierarchy of ordered wholes, ranging fro
However, one must also view this universalistic approach to wholism with an element of caution. Such a perspective lends itself to an infinite regression that could lead the wholist to the state where understanding anything would require that he explore everything in larger and larger context. This view could leave him in a position of not being able at the same time to understand and cope with the particulars with which he began - a position that would be counterproductive to a wholistic perspective. And we have seen the remarkable effectiveness of the Reductionists’ efforts to explain the world around us!
Complexity and Chaos: Mathematics when applied to the laws of physics enables one, generally, to predict the cause and effect of natural phenomena. In fact, one of the remarkable features of applied mathematics is that it performs this function so universally and so well. The times and locations of the next century’s solar eclipses and the structural integrity of a yet-to-be-built bridge can both be determined very accurately, for example. Predictability is the essence of science, and the mathematics derived for the laws of nature enable these predictions. The success of this symbiosis is principally due to the good fortune that the majority of the basic laws of nature are deterministic, at least so within an acceptable degree of approximation. Given known input parameters and the appropriate equation, one can determine the resulting behavior of a system. Determinism is so implicit in the Western science paradigm that it has been only in the past few decades that physicists and mathematicians have recognized underlying unpredictable behavior within deterministic systems, while the constituent parts of these same systems are governed by deterministic equations. It was universally thought that predictability could be universally achieved if only sufficient information could be gathered and processed. This viewpoint was held until the early 1960s when Edward Lorenz discovered that simple deterministic systems with but a few elements could generate very erratic behavior. In addition, this apparent randomness was fundamental in that no amount of additional information refinement could make it go away. Furthermore, the enigma of a seeming paradox revealed this “randomness” also to be deterministic, generated by rigorously determined natural laws. In principle, all phenomena are predictable; in practice, small uncertainties can be so dramatically amplified as to render long-term prediction impossible. This randomness generated in this way has become a new science and is called “chaos.”
Chaos applies to systems that operate within the parameters of known laws, but their behavior is to all appearances impossible to predict. For instance, what do epidemics, the weather, the stock market, the California coastline, orbits of stars, population growth, and the great red spot of Jupiter all have in common? Each is an example of a chaotic system. [6]
Possibly the first early glimpse into the dynamics of chaos was observed by the luminary French mathematician, Jules Henri Poincaré. The king of Sweden had sponsored a contest to show rigorously that the solar system as described by Newton’s equations of motion is dynamically stable. [7] Poincaré recognized this challenge as a generalization of the classical three-body problem, considered to be one of the most difficult problems in mathematical physics. Poincaré picked up the gauntlet and proceeded toward a solution. While he was not successful in obtaining a complete solution, his approach was so elegant that the king awarded him the prize anyway. The difficulty of the three-body problem is showing that a solution converges in term of invarients for the nine simultaneous differential equations. A solution to a multi-body problem is many times more difficult. As a result of this experience, Poincaré developed an insight into sensitive dependence on initial conditions, which he expressed in his own words in 1903 as:
|
“A very small cause which escapes our notice determines a considerable effect that we cannot fail to see, and then we say that the effect is due to chance. If we knew exactly the laws of nature and the situation of the universe at the initial moment, we could predict exactly the situation of that same universe at a succeeding moment. But even if it were the case that the natural laws had no longer any secret for us, we could still only know the initial situation approximately. If that enabled us to predict the succeeding situation with the same approximation, that is all we require, and we should say that the phenomenon had been predicted, that it is governed by laws. But it is not always so; it may happen that small differences in the initial conditions produce very great ones in the final phenomena. A small error in the former will produce an enormous error in the latter. Prediction becomes impossible and we have the fortuitous phenomenon.” |
What was remarkable about this observation was that Poincaré made it during a period in history when deterministic physics was the established dogma.
Chaos and complexity are integrally interrelated. If complexity is the thunderstorm, then chaos is its fury. A system is chaotic when its behavior is sensitively dependent upon its initial conditions; or in other words, it is a system where minutely small perturbations can produce very large effects. The concept of complexity is a paradigm shift from the classical deterministic and reductionist philosophy of the Newtonianists to a more probability-based and wholistic worldview. Although this paradigm shift is already occurring in some areas of physical science, Prigogine and Stengers caution that breaking from the spell of the machine paradigm as the gold standard for physics in particular and science in general will be very difficult, but essential because of the limits of reductionism. [8] A case in point occurs in quantum physics where the objective properties of a particle can not be deduced through reductionism. Because of limitations such as the Heisenberg Uncertainty Principle, one can only determine its probabilities. If a particle’s momentum is known, its position is unknown; and conversely, if it can be located in space, its momentum can not be known. One can predict only the probability of the alternative state. Probability itself is a fundamental property of nature! [9] Physicists only in recent years have become aware of the intrinsically unpredictable manner in which systems of classical mechanics can behave. While they are deterministic in principle, they can be unpredictable in practice. The simplest systems described by the simplest equations can have chaotic solutions. Furthermore, these same systems can behave predictably or chaotically as a reaction to a minute change to a single term in their equations. Hence, many natural processes may be explainable through chaos theory. The abrupt transition between laminar to turbulent flow in fluids is one example.
Chaotic systems themselves are not random. They have patterns of self-similarity and an aggregate predictability. Contrary to general notion, chaos is not really the opposite of order, although that definition is acceptable is some contexts. Chaotic systems dwell in the gray twilight between order and disorder. They follow rules, but the simplest of rules can produce profound complexity. The slightest change in an initial condition will result in a radically different solution than would be experienced had it been left untouched. Chaos theory is not about disorder, nor does it reject determinism. On the contrary, the emphasis of chaos theory is not on the inherent unpredictability of a system, but on the inherent order of the system as the universal behavior of similar systems. Chaos is about complexity. It is about nonlinear, dynamic systems that are not in equilibrium – the world of our daily encounters. The apparent paradox of chaos theory is that while it predicts that complex nonlinear systems are inherently unpredictable because of sensitive dependence upon initial conditions, it offers a way to express these unpredictable systems through representations of the system behavior.
With fractals and
phase-space diagrams replacing the traditional Cartesian system for
interpreting complex systems, chaos theory now enables science to look at
nature with a dramatically different paradigm from that which has domi
Chaos
provides a natural complement to the Second Law of Thermodynamics. The Second Law is the theorem on the
degradation of energy. It affirms that
all forms of energy are not equivalent, and the energy quality of a system and
its surroundings can never spontaneously increase – only decrease. The Second Law, the law of ent
Yet Nature abounds
with highly ordered, complex systems continuously coming into existence. The randomness and disorder created by the
ever-increasing ent
Characteristics
of Complexity. What, then, is
complexity? A rigorous definition is
elusive. Complexity, it can be said, is a state that lies somewhere between
order and disorder – a state at the “edge of chaos.” It is neither random nor is it unresponsively
static. Complexity is a property that is
often applied indiscrimi
As discussed earlier, there is a considerable bias toward simplification encouraged not only by the scientific method in the study of Nature but also in our social and intellectual environments. Complexity, however, addresses the perspective that reducing complex multidimensional issues to a linear either-or decision process would result in invalid conclusions, and therefore it offers alternative forms of handling problems in a manner such that they can be better dealt with. But as its name implies, complexity comprises numerous attributes, and several researchers have attempted to identity these various characteristics. [10-12] On the macroscale, complexity involves conventional systems, as well as organic and connectionist perspectives. The system perspective as it pertains to complexity is interdisciplinary in that it addresses generalized or abstract systems in search of common properties across various forms of organization. This concept of systems is often associated with cybernetics, a self-contained and self-regulating entity that incorporates feedback. And again, parts are not looked at in isolation, but instead the wholistic function of the system is considered. This perspective also relates to emergence as new system properties are generated that contain functions not found in any of the parts.
The organic system perspectives of complexity are either biological or bio-like in their functions. They are capable of both self-replication and self-repair. They not only respond to their environment, but they also adapt to it and change their behavior as required. Because of the often-stochastic interactions of the various system parts, the control of organic systems is decentralized, which differs from that of traditional systems. These attributes are characteristic of the computer-simulated life forms undergoing extensive study at the Santa Fe Institute as a major thrust within their Artificial Life, or Alife, program. [13, 14] Their simulations study the creations, evolutions, interactions, adaptations, and extinctions of artificial species or ‘agents’ that live only within the computer program. The results of these studies have contributed much to the understanding of complexity. Connectionism takes its cue from the cognitive sciences with their neural system connections within a brain-like structure. One of the defining features of the connectionist systems is that they self-organize. (Self-organization is discussed in the next section.) The communication of information across the system and the feedback of this information allowed by the connections can cause the formation of attractors, which in time can stimulate self-organization. Since a connectionist system is defined by its connections and not by its parts, it may operate in static, chaotic, or organized modes – or combinations thereof.
The prototypical complex system normally comprises all three of the macrosystems described above. The effects of this synthesis is the creation of a self-organizing complex system with the following characteristics, as suggested by several complexity researchers: [e.g., 10-12]
Attractors: These are alternative areas of stable operation or concurrent options that admit several possible simultaneous behaviors. The initial conditions and subsequent system history determine which action will actually occur. Attractors are necessary for system self-organization.
Autonomous Components: These are the independent agents typically comprising complex systems. All are equally valuable in the functioning of the systems and control is decentralized, implying that there is by design no dominant component directing the actions of the others. Since control cannot be imposed, any centralized structure that may be required by the system will emerge through self-organization.
Coevolution: The parts of a system evolve in conjunction with each other so as to fit into a wider system environment. How well various parts may “fit” in their environment is determined in the context of the dynamics of the current niche. This environmental dependence contrasts with conventional science’s tendency toward isolated treatments.
Downward Causation: In complex systems the properties of the parts can be affected by the system’s emergent properties or the higher-level system qualities of the whole. This phenomenon can induce constraints on the flexibility of the constituent parts. An example of this might be a psychosomatic illness where the whole body affects the function of a specific organ.
Emergence: When a complex system develops properties that are not describable in terms of their parts, these new properties are said to be emergent. These higher-level functions comprise synergistic organizations that exceed the simple notions of aggregation advocated by the Reductionists.
Fitness: The structure of a complex “surface” comprises numerous local maxima and minima modeled as a contextually dependent fitness landscape. These “hills” and “valleys” depict the modeling choices, wherein the heights and depths of these hills and valleys relate the value of the option.
Fuzzy Functionality: This comes into play when the overall system function is not known initially. Instead, the system function is created by co-evolutionary methods of probabilistic matching between the system and its environment in the absence of well-defined classifications.
Instability: This contrasts with the steady-state models of conventional science in that “catastrophes” are both possible and admissible. This could come about with the sudden change of a system attractor.
Mutability: New configurations become possible due to random internal structural changes occurring within the system. Parts of the system may be created, destroyed, or modified. A biological example would be a mutation.
Non-Equilibrium: Ilya Prigogine has stressed in many
publications [e.g., 8] that complex
systems are dissipative and hence operate far from equilibrium. Energy derived from the environment forces
the system away from a state of equilibrium and to a semi-stable state that is
far from equilibrium. A living organism
would be a typical example. A feature of
these active systems is their ability through this imported energy to reduce
their local ent
Nonlinear: Complex systems are not linear. Hence, the mathematical process of the superposition of parts (where for example f(x + y) = f(x) + f(y)) so often applied in reductionism is no longer valid. The whole of a complex system is different from the sum of its parts, implying that the superposition of the properties of the parts will not produce a valid solution for the whole. The system must be analyzed wholisticly.
Non-Standard: Complex systems tend to be heterogeneous. Because their various parts are free to vary their associations, “clumping” and changes over time are both permitted. Hence, systems that are initially homogeneous will likely develop self-organizing structures and merge to a state of heterogeneity.
Non-Uniform: Each part of a complex system may evolve separately. This admits non-uniformity and allows for diversity in the local rules each obeys, rather than requiring that all parts follow the same global law. The occurrence of this mix of rules is dependent upon the system’s overall contextual coevolution.
Phase Changes: Complex systems involve feedback processes, which can lead to sudden changes in system properties called phase changes. In classical thermodynamics the sudden change from a liquid to a gaseous state is a phase change. The precarious “edge-of-chaos” status of a complex system represents a phase-change critical point for which the system’s self-organizing dynamics maintains it at the phase boundary.
Self-Modification: Insomuch as the parts of a complex system can freely change their connectivity, either randomly or through learning processes they may have evolved, the system can redesign itself over time. This action, of course, is constrained by the system’s need to maintain its operating function or to modify its behavior to conform to an environmental change.
Self-Reproduction: Some complex systems have the capability of cloning either identical or edited copies of themselves. While biological examples are obvious, social systems such as fast-food franchises can replicate additional similar systems. In addition, new system structures are made available through copying errors, such as mutations, recombinations, or insertions, thus allowing open-ended evolution and self-generation.
Undefined Values: When the context of a system’s environmental interface is not initially specified, then it will evolve dynamically through environmental interaction. This dynamic creation of semantic communications is not a direct mapping of the world outside the system as is usually assumed.
Unpredictability: This parameter is characteristic of the mix of attractors often present in complex systems wherein the system may display a chaotic sensitivity to initial conditions.
While these characteristics come into being as a natural adjunct to complex systems, note that none of these is incompatible with what is understood as “conventional” science. Any conventional system state can be restored by imposing a set of boundary conditions required to reduce the scope of the system studied, thus constraining the system behavior to that of classical Newtonian physics or to one at the statistical end of the spectrum. But when this step is taken, there is a richness in the understanding of Nature that is lost. As we so pretentiously asserted in Chapter 1, biology is constrained by chemistry and chemistry obeys the laws of physics. But biology does not reduce to chemistry, nor does chemistry reduce to mere physics. The richness of a single-cell amoeba, while comprised of a multitude of atoms and molecules, is not explained or described by these atoms and molecules. It is something more. It is an emergent entity wrought by complexity.
Perhaps one of the better ways to obtain insight into the system characteristics of complexity is to review how it has been applied to brain neural functions. And since this thesis suggests that there is a strong parallel between an evolving complex neural computer and the mammalian brain, a discussion of this nature would be most relevant. During the past decade, there have been numerous attempts to quantify complexity with mixed results. But possibly one of the more complete versions was developed by Tononi, et al. [17]
The world of neuroscience has for years debated the views of the “localizationist” against those of the wholist in regard to brain function. The localizationist maintains that brain organization stresses specificity and modularity, while the wholist emphasizes its global functions – complete with its mass action and Gestalt phenomena. [18] Yet both of these contrasting functions coexist in the mammalian brain. Different brain regions are functionally segregated, yet all regions are integrated in their perception and behavior. First, the brain is not only functionally segregated, but it is functionally segregated at multiple organizational levels. Local collectives of strongly integrated neural groups have formed out of developmental events and activity-dependent selections dictated by the habits of the particular mammal. Each group has a tendency to connect to a specific subset of other groups and together to specific sensory nerves, showing preferential responses to different stimuli. Furthermore, there is further refinement in functional segregation for different attributes of the stimulus, such as color, form, and motion. And neurologists have observed countless examples of loss of very specific functions as a result of localized cortical lesions.
But at the same time, global integration of brain activity is observed at levels ranging from the neuron to overall behavioral output. The cortical-pathway architecture is such that any two neurons at any location are assured of a separation from each other by only a few synaptic steps. Furthermore, the majority of pathways linking any two areas enable two-way information flow, thus providing a structural substrate for reentry into each area. [17] Reentry can be thought of as the cooperative interactions within and among functionally segregated brain areas as mediated by a process of ongoing, parallel signaling. [19] Such linkage enables a process of continuous recursive signaling among such groups across massively parallel paths. Widespread patterns of correlations emerge among neural groups as one of the dynamic consequences of this reentry. As a result, one experiences unified and globally coherent perceptual scenes in the case, for example, with vision. Such coherence is an essential property for the unity of behavior. Experiments with the disruption of these various cortical areas have often demonstrated discontinuities in the integration processes.
Tononi, et al. [17] have demonstrated that the constructive and correlative properties of reentry lead to the natural rise of a balance between the functional segregation of specialized areas and their more global integration and that the two organizational areas can be represented within a unified framework. The results of this research has enabled them to derive a mathematical expression reflecting this interplay between functional segregation and integration within a neural system – an expression referred to by Tononi, et al. as “neural complexity.” If neural systems are taken to consist of a collection of elementary components (brain areas, neuron groups, or individual neurons), then the level of neuronal groups can be chosen and their dynamic interactions can be studied based on their interconnection topology. The statistical properties are assumed invariant and the neural connectivity remains fixed.
What was observed is the following. When different neuronal groups are considered a few at a time and the activities of these neuronal sets tend to be statistically independent, then these groups are functionally segregated. Groups showing a high degree of statistical dependence when considered several at a time, on the other hand, are functionally integrated. The interplay between functional segregation and integration within a neural system is the neural complexity. It would be beneficial to see how this relationship is derived and to discuss its meaning.[1]
Let us consider a
neural system X comprised of n
neuronal groups. Following the
suggestions of Papoulis [20], we can
assume that a stationary, multidimensional stocastic process can describe this
activity and that the joint probability-density formation can be characterized
in terms of ent
Ent
Now let the
system’s deviation from independence be represented by “mutual information,”
identified as MI, and partition this system X into two parts: One will be the jth segment, the subset comprising k components, represented as Xjk. The other will be its complement, X - Xjk. The mutual information between Xjk
and its complement as a function of ent
MI(Xjk; X - Xjk) = H(Xjk) + H(X - Xjk) - H(X), (1)
where H(Xjk)
is the ent
Now let us
generalize the concept of mutual information so as to determine the deviation
from statistical independence among the n
components of the system. This can be
done with a single parameter referred to as its “integration,” identified as
I(X). The integration parameter is
defined as the difference between the sum of the entropies of all individual
components of the system when they are considered independently and the ent
n
I(X) = S H(xi) - H(X), (2)
i
= 1
where xi is the individual ith component.
If we now convert to the two-partition system through combining and rearranging Equations 1 and 2, the integration becomes
I(X) = I(Xjk) + I(X - Xjk) + MI(Xjk; X - Xjk). (3)
I(X) will be equal to the sum of
I(Xjk) and I(X - Xjk) only when the
system components are completely independent, representing the state of maximum
ent
Let us now consider subsets of the system (X) comprising k-out-of-n components that we will represent as Xk. The average integration for these k-size subsets can be defined as {I(Xjk)}, where j indicates that an average is taken over all n!/(k!(n - k)!) combinations of k components. Also note that when k equals n, {I(Xjn)} equals I(X), and this same term equals zero when k equals one. We can further note from Equation 3 that {I(Xjk)} increases monotonically with k.
Now let us define the complexity of the system as the difference between the values of {I(Xjk)} expected from a linear increase for increasing subset size k and the actual discrete values observed. Hence, complexity can now be written as
n
Cn(X) = S[(k/n)I(X) - {I(Xjk)}], (4)
k
= 1
where Cn(X) is either zero or positive.
Referring back to Equation 2, we can also express the complexity as a function of entropies, which would become
n
Cn(X) = S[{H(Xjk)} - (k/n)H(X)]. (5)
k = 1
In addition, as we observed from Equation 3, the complexity also corresponds to the average mutual information between the two partitions of the system as everything is summed over all bipartition sizes (i.e., summed to n/2). Complexity then relates to mutual information in the following manner:
n/2
Cn(X) = S{MI(Xjk; X - Xjk)}. (6)
k
= 1
The beauty of this relationship derived by Tononi, et al. lies in its elegance and simplicity. Although these mathematical expressions were derived for neural systems, this notion can be generalized and applied to other complex systems as well. However, we must recognize that in the derivation of Equation 6 there were dramatic simplifications and linearizations applied in order to conveniently handle the mathematics. These results are only approximations that we may use to get a feel for the behavior of complexity. A real neural system is much more complex – and nonlinear. But nonetheless, if we look at what has resulted, we observe that the complexity is high when the mutual information between any system subset and its complement is high. And as one would intuitively expect, a necessary condition for high complexity within a neural system is a state of very high interconnectivity with neural groups synchronizing into ever-changing configurations.
If one were to
generate a computer solution for Equation 6, it would produce a family of
curves similar to that depicted schematically in Figure 4.2. [21]
Curve A depicts a state of high complexity representing a balance
between the mutually independent firing of specialized groups of neurons during
functional segregation and the coherent joint activity or integration of these
groups. Curve B represents a significant
decrease of connection density that results in a more independent firing of
individual neuron groups, causing desynchronization. The net result is much lower complexity. Curve C would result from a uniform
connection among the various neural groups, causing the majority to fire
synchronously and independently of response properties. Because of this uniform behavior, not only is
the integration low, but there is no functional specialization of the different
neuron groups. Therefore, the
complexity is also low. Curves B and C
correspond to the ![Text Box:
Figure 4.2. Depiction of Equation 6 for three widely varying cases. [21]](Chapter4_files/image004.gif)
gas and crystal-lattice analogies,
respectively, shown in Figure 4.1.
As man-made systems become more complex and as we extract more and more design parallels from Nature, we find we are less constrained by classical paradigms and are able to explore these new principles and apply them to our ever-evolving technology with a greater understanding of the whole.
Self-assembly. The idea that any system would be capable of assembling itself likely rates with the concepts of antigravity and time travel in the minds of most people. The human paradigm for putting things together has changed very little in principle since the first hominid hammered pieces off both sides of a flint to make a tool. Albeit our processes are much more refined, we still fabricate things predominantly by removing material to produce the desired result. Ours is basically a “top-down” approach where we start with a nondescript billet and shape it to what we want. Nature, on the other hand, has chosen a “bottom-up” approach toward assembly, and furthermore has designed its systems so that they assemble themselves. Examples of this are all around us. On a macroscale, the cohesive forces of condensed water vapor cause it to coalesce in the air to form clouds. The presence of gravity carves the landscape by forming mountains and cutting streambeds. Ecosystems emerge as self-organizing entities. But it is the microscale or molecular self-assembly in which we are most interested, and that includes all of biology.
Self-assembly, self-organization, or self-replication, as either name implies, is the ability of a system to configure itself solely through the assistance of a precoded “blueprint” and resources from its environment. There is no active involvement of humans, as this migration of molecules into a functioning ordered entity all takes place without human intervention. Raindrops that fall onto a leaf spontaneously form circular globules with very smooth surfaces because the laws of thermodynamics require that the energetic stability of each droplet be maximized. But only the simplest structures are constructed by thermodynamic self-assembly. The self-assembly of a complex living system is a different matter. The biological mechanism for this phenomenon is poorly understood and is a growing field of research. When a single-celled zygote first divides on its way to producing a complex being such as a human, what information bits cause what actions to form which structures? What determines how and where fingers will form or how the brain will form? Detailed understanding of self-assembly at the cellular level may likely be a few years away. But once discovered, these principles will certainly precipitate a revolution of their own.
The economic success of nanotechnology, and more specifically nanoelectronics, will depend upon the development of successful self-assembly techniques. As we saw in Chapter 2, the production of nanoelectronic devices by fine lithography is rapidly approaching its lower practical limit, and further miniaturization is beginning to severely drive the costs of the integrated-circuit fabrication facilities. According to Moore’s “Second Law,” these manufacturing facilities, which currently cost several billion dollars, will double with each new generation of smaller, more capable devices. The expensive lithography required to form the ever-smaller features, coupled with the demand for an ever-cleaner environment to avoid defects, constitute the overwhelming cost drivers. To avoid this need for fine lithography as well as to investigate innovative nanofabrication processes, numerous self-assembly techniques are being explored. [22-29] The basic concept of self-assembly, when properly applied, is to take advantage of natural forces with minimal energy to produce the desired device. In Nature, this is a wholistic process. In a human-controlled environment, successful self-assembly will depend heavily upon its remaining so.
The self-assembly of nanoelectronic “wires” and devices offers a myriad of extraordinary advantages, among which are the following: First, it follows a biological parallel that can draw upon a virtually unlimited abundance of examples for guidance. And as further research opens more doors into the biomechanisms of self-assembly, then this newly derived knowledge can be immediately applied to a synthetic system. Second, molecular self-assembly begins at the atomic level of the structure and accomplishes the most difficult steps in nanofabrication of configuring and modifying the nanostructure, typically through the use of synthetic-chemistry techniques. Third, since self-assembled structures tend to be thermodynamically the most stable by possessing the least free energy, they tend to be self-healing and fairly defect-free. And fourth, a self-assembled system can directly incorporate a biological structure into its final configuration, if such integration would prove to be an advantage. An example of this might be the mass production of a certain nanoelectronic device through modification of the DNA of a specific strain of bacteria in much the same way a virus might be reproduced.
But
until the biomechanisms of self-assembly are better understood, several
alternative approaches for which the physics is more clear are being
investigated, and a short discussion of these efforts is warranted. Ayres [30]
makes a strong argument for self-assembly as a manufacturing strategy by
suggesting that the measure of an entity’s distinguishability from its
environment can be couched in terms of information theory. The application of information theory to
natural or man-made systems could aid considerably in discovering the general
rules for the design and fabrication of the desired computing architectures,
given knowledge of information storage, data processing, and communication
within these systems. Information theory
could likely provide a powerful resource for deriving a better understanding of
the fundamental driving forces of self-organization. Ayres classifies information metrics, such as
ent
If
one thinks of the universe as serving as the ultimate information source, then
a truly “intelligent” system would be one capable of gathering, storing, and processing
information from its environment.
Systems unable to perform this function or to control the flow of
information would tend toward disorder and thermodynamic equilibrium. Thermodynamic equilibrium is a state of
minimum energy and maximum ent
The
way molecules configure themselves and hence influence the properties of space
is a source of other governing information.
Snowflakes all have six points, although no two are allegedly
alike. The proper balancing of the
repulsive and attractive interactive forces within a crystal lattice causes the
atoms to organize to form ordered solids.
The information received by each atom is processed and transferred to
the other atoms until perfectly defined symmetry is obtained. However, a snowflake is a much more complex
system than is a crystal. Although it is
a perfectly self-organized system, it has increased its structural complexity
by adding new information extracted from its environment. This information is in the form of
nonequilibrium factors from the temperature, pressure, and wind within which it
dwelled when it was created. Certain
polymers, referred to as dendrimers, are capable of configuring into
well-ordered sph
There is little doubt but for what material self-assembly will become the primary manufacturing process for nanoelectronics components as the knowledge base of self-organizing mechanisms continue to evolve. As discussed above, we saw how the CMOS process utilizes very complex lithography to overlay ever decreasingly small patterns onto silicon-dioxide chips to produce today’s microprocessors. We also saw how this ever-decreasing miniaturization is becoming a major cost driver, which is in effect the necessary increased cost of morphological information needed to prevent defects. Even such precision manufacturing suggestions as the direct positioning of atoms with nanoprobes, while appearing elegant, pale in the light of the required tooling costs – estimated at around $15 million per patterning tool with a need for up to 50 tools per fabrication. With the present trend of CMOS patterning costs escalating dramatically, other nanofabrication approaches are being seriously investigated. Once self-assembly becomes practical and routine, especially when applied to nanoelectronics, then engineers need only to maintain the flow of energy and material through the system while the nanodevices build themselves. Self-assembly, then, can be taken to be the generation of complex structures from elemental units with a minimal flow of morphological information.
Zhirnov and Herr classify self-assembled systems according to the type of elementary unit whence they are derived, whether it be atom, molecule, or cluster. When atoms are ordered as a result of a physical deposition process, such as chemical-vapor deposition or molecular-beam epitaxy, this process is called physical self-assembly. Research in physical self-assembly attempts to take advantage of nonuniform imperfections that exhibit unique properties that may be useful when applied to nanoelectronics. For example, experiments with high-energy electron radiolysis to induce anionic vacancies in calcium fluoride crystals in an attempt to form a superlattice within the host material may find a potential application in the field of electronics. The concept of smart templates is another example of physical self-assembly. Predetermined variations in a substrate’s diffusivity, reactivity, bond energy, and surface energy can generate a smart template to produce self-organized nanostructures. Control of such physical features as surface discontinuities, interface stresses, and disposition kinetics also help develop the driving forces that guide self-assembly processes. One example of a smart template is a substrate surface that has been strained at locations where specific deposits are desired. The deposited material preferentially moves to and bonds with the strained areas. Nogami has demonstrated the self-assembly of a single-atom metal line (or “nanowire”) through the anisotropic diffusion of indium on a silicon surface. [33]
When molecular-scale compounds are ordered with precisely designed atomic architectures into more macroscopic structures, this is referred to as chemical self-assembly. Chemical self-assembly likely holds the greatest near-term promise for a practical large-scale production of nanoelectronics. Absolutely identical molecular configurations can be formed spontaneously, and the desired structure and its properties can be tuned through control of the chemical synthesis. Chemically self-assembled structures are not only fairly easy to prepare, but they also take shape quickly from emulsions of the elemental molecules. Furthermore, the structures are not only robust and molecularly ordered, but as mentioned earlier they are also thermodynamically stable and tend to have negligible defects. The polyphenylene molecular wires discussed in Chapter 2 were produced by chemical self-assembly. The approximately 1,000 identical three-ring polyphenylene-based chain molecules were produced in parallel from the tips of gold contacts with thiol groups (i.e., sulfur hydrogen) providing the interface between the molecular chain and the gold tip. The sulfur from the thiol group absorbs well into the gold substrate, thus forming a firm anchor.
Processes characterized by nanoparticles aggregating into clusters comprise the third category of self-assembly, and this process is called colloidal self-assembly. Structures derived from colloidal self-assembly demonstrate hybrid properties different from either individual atoms or bulk solids and may demonstrate controllable electronic transport properties, thus enabling novel electronic devices. Work in this area, however, is still in its embryonic stage.
The self-assembly of complex structures, especially entities with a very high degree of complexity as represented by a life form, will require some form of imbedded information to augment the assembly process. [34] This information supplement is referred as coded self-assembly, as all instructions for fabrication of the system would be integrated within the individual components whence the system itself evolves. The coupling of the promising features of chemical self-assembly with encoded instructions that would enable a complex structure to assemble itself in a manner mimicking that of DNA is extremely attractive. Research into the understanding of coded self-assembly has only begun, but its potential staggers the imagination.
Complexity, as we have seen, is an integral characteristic of Nature, the source of its richness and wonder. As humans, we have sought to harness the powers of Nature by cleaving and reshaping its basic constituents; in other words, by reducing it to its basic elements to understand its function - the reductionist's approach of classical physics. But Nature's most intriguing offerings are very complex and understood only when viewed as a whole. The new science of complexity offers a unique insight into this wholistic perspective of Nature. And complexity will also experience an emerging significance for man-made systems as more sophisticated systems are brought about through advancements in nanoelectronics. As we will see later, the characteristics of complexity that lead to the edge-of-chaos order essential for living organisms also form a necessary condition for synthetic sentience. Sentience is a wholistic phenomenon that is not predictable by the reductionist's analysis of its individual parts. Classical reductionism will not explain its emergent behavior.
The nanoelectronic devices and structures required for a synthetic-sentient entity will be "grown" in the same manner as that for a natural system, a seemingly overwhelming practical constraint for a man-made system. But just as one chapter closes, another opens. Discoveries of assembly processes not possible in the old chapter will become routine in the new. The approaching end of the line for Moore’s Law in CMOS technology has led us to a new pathway into a realm that will prove to be more dramatic and exciting than the one just traveled. And like the Phoenix rising from its ashes, Moore’s Law will live on in this exciting new domain. Self-assembling nanostructures are at the juncture today that CMOS technology entered in the early 1960s. This new technology will be disruptive and overwhelming as computing systems become ever more complex from the ever-increasing infusion of nanoelectronics. But again we need only to turn to biology for guidance. There is no state of complexity greater than that of a living organism. Yet all of life’s mysteries, all of the wonders of science and knowledge of the universe, are achieved through the permutations of only four basic molecules that comprise DNA. Tomorrow’s most complex computers will in kind evolve from a few standard components. Entire subsystems will be self-assembled, and the emerging complexity of the complete system will take on characteristics barely imagined today.