Dr Jacques COULARDEAU

JEFF HAWKINS – ON INTELLIGENCE ­– 2004

This is an extremely important book in the field of Artificial Intelligence. The author reject this Artificial Intelligence because it identifies intelligence to the behaviors produced by this intelligence. Hence the machine simulates intelligent behavior but is not intelligent. Three things are essential goals to satisfy if we want to move towards intelligent machines. We have to take into account and integrate time. We have to include as architecturally essential the process of feedback. We have to take into account the physical architecture of the brain as a repetitive hierarchy. Strangely enough the main mistake is already present in this first programmatic intention. Jeff Hawkins does not include the productions of that intelligent brain. I mean language, all ideological representations or models of the world from religion to philosophy and science, not to speak of arts and culture. And strangely enough this mistake is locked up in an irreversible declaration:

“A human is much more than an intelligent machinre . . . The mind is the creation of the cells of the brain . . . Mind and brain are one and the same.” (41-43)*

We cannot but agree with the first sentence, but the mind is not “created” by anything. It is produced, constructed by the brain from the sensorial impulses it gets from the various senses and the way it processes them in its repetitive and parallel hierarchical architecture. But the mind is a level of human intelligence of its own. Unluckily Hawkins will not see it. I have already said what it excludes from this human intelligence, but we must add the fact that this human intelligence lives in a situation that enabled this intelligence to develop and invent its first tools when Homo Sapiens started its journey on earth some 300,000 years ago. This situation requires from the weak animal that Homo Sapiens is to develop these tools to compensate for its weakness, and to coordinate its survival and development with communication and social organization which implied and required a culture, a model of the world to migrate, develop new productive means, and be able to develop as a species in order to expand all over the world: Homo Sapiens was a migrating species from the very start because of his very brain and the mind it could procude. Jeff Hawkins forgets about the phylogeny of Homo Sapiens. He takes intelligence as existing in itself without a genesis from nothing to what it is today. In other words he speaks of evolution but he does not study it and how this evolution brought this human species into developing intelligence, means of communication and means of production that did not exist before.

At the same time he does not consider the feelings and emotions of that human being and he at best locates them in the old brain, the brain inherited from the species before mammals since the cortex only developed with mammals. It is also obvious this is a mistake. Due to mirror neurons man is able (with some top mammals along with him, to develop empathy, the possibility to imitate (hence to learn through imitation and when language was invented to learn through repetition) and to share the feelings of others and one’s own feelings with others. It is this ability more than the old brain that is at stake here and is neglected. That makes Hawkins neglect social aims, productive objectives, cultural targets, ideological psychological social motivations and of course social organization. To invent and develop intelligent machines would not even exist as a plan or a project or even a desire if Homo Sapiens had not been able to blaze and then run the track leading to development.

He is sure right on the fact that behavior is only the consequence of all this but by rejecting behavior because he rejects behaviorism (which is purely ideological on his part) he also locks himself out of the possible approach of human relations, human motivations towards others, hence concrete, material and also emotional and intellectual behaviors. And that prevents him from coming back to the situation that has to be controlled and set up collectively to reach collectively defined objectives. Globalization is right now the best example of how objectives have to be defined at the level of the planet and no longer at the level of particular countries or groups.

But apart from that the whole book is essential because Hawkins concentrates on the study of the brain and its hierarchical architecture, and I should say its double architectural structure, not double in nature but double in working.

The whole adventure starts with the senses and he straight away says there are a lot more than five senses even if we can consider there are only five basic sensorial organs: the eye, the ear, the tongue, the skin and the nose.

At the level of the eye we have to add motion, color, luminescence and spatial orientation. At the level of hearing we have to add pitch, length, intervals, timbre, spatial orientation and balance (vestibular system). At the level of touch we have to add pressure, temperature, pain, vibration but also spatial orientation and movement on the skin that will be useful both in torturing (along with pain) and eroticism or emotions (along with pleasure). At the level of smell we have to consider intensity, appeal (good, bad or somewhere in-between), spatial orientation. At the level of taste we have to add temperature, texture, appeal (good, bad or somewhere in-between), and even finer elements like sweet, salty, acid, alcoholic and many others. But, and he insists on that, the general senses of the body are essential too. The whole body is a network of sensors that checks and measures our joints and joint angles, all our bodily ,positions, and all proprioceptive receptors (sensory receptors, in muscles, tendons, joints, and the inner ear to detect motions or positions of the body or the limbs, that respond to stimuli arising within the body.) Note these are indispensible for walking, running, swimming and all movements, particularly coordinated movements like gymnastics and all kinds of martial arts And we should add the physiological sensors and mechanisms that measure our inner level of satisfaction, dissatisfaction, balance and unbalance of every single organ of ours. These last sensors are essential for a new born child since it is those he/she will use from the very start and that will prompt his first cry or call. And every single of these senses and sensors sends messages to the brain in temporally organized sequences. The eye reboots its vision three times per second, what is called a saccade.

The first hierarchy he takes is exemplified by vision. I will integrate the eye into it right away though the eye is more or less marginalized in Hawkins’s approach. And here the eye sends many messages according to the particular abilities of the various retinal cells that capture the signal. I will insist on the fact that he neglects: the signals are sent from the retina and are spatially oriented right-side right and upside down. He neglects it because we do not have an “image” on the retina and it is not an “image” that the retina sends. But the spatial orientation of this “pattern” as he calls it is essential. The brain will have to interpret this orientation to reestablish the proper one thanks to the signals sent to the brain by the other senses and thanks to its experience starting right after birth. Experiments have been performed using glasses that inverted the orientation of the “pattern” on the retina and after a short while the brain corrected the initial correction and provided the mind with the proper spatial orientation.

In the neocortex, the capture of a visual stimulus is hierarchically organized and we must keep in mind that the signals are renewed three times a second. In the V1 area only many small segments and isolated characteristics like colors are deciphered. These numerous small elements are sent to the V2 area where they are regrouped into larger elements. Then they are sent to the V4 area where they are regrouped into recognizable elements like a nose, an eye, etc Then they are sent to the IT area where they are reconstructed into a face for example. Here Hawkins defines a pattern as being “a stable cell assembly that represents some abstract pattern” (p. 80). At each level after learning, hence after first stimulation by one unknown element (which is sent unanalyzed to the hippocampus that takes over, identifies it and sends it back into the system), an invariant representation of each identified pattern is memorized (cortical memory, p. 100) in the cells (he does not specify the electrical and chemical procedure nor the molecular level of it). The cortical procedure then, after learning, is a recognition procedure: the pattern received corresponds to one invariant representation previously memorized, otherwise it is sent up as far as the hippocampus if necessary. The last element we have to understand is that the identification is not done in detail but as corresponding to an invariant sketch of the element and that sketch accepts variations. That explains why we can recognize someone and yet be mistaken. The mind did not make a mistake it used some elements that corresponded to the sketch it had in memory, and that was the wrong sketch.

The three basic characteristic of this hierarchical functioning are:

1-                          its sequential memory (sequences of patterns hence spatial in the pattern and temporal because serialized);

2-                          its autoassociative nature (it memorizes a sketch and not the real detailed pattern when learning, though this detailed pattern is also memorized which enables us to realize we made a mistake when we took someone for someone else, and then it recognizes this sketch in the real pattern it receives after learning);

3-                          and finally its “invariant representation” dimension which is the identification of these sketches as referents for further use. Here instead of saying that these sketches have to be “named” he should have said that they have to be identified at each level with some kind of Cortical Identity (CI) and this when connected with the invention of language by Homo Sapiens, or the learning of language by children would have led him to the word “concept” that he uses rarely, and the operation of “conceptualization” that he does not use at all. Homo Sapiens seems to be the only animal who managed this conceptualization power of the neo-cortex (dominated by the hippocampus) into producing language.

We come then to the heart of the volume:

“The three properties of cortical memory . . . (storing sequences, auto-associative recall, and invariant representations) are necessary ingredients to predict the future based on memories of the past . . .  Prediction . . . is the primary function of the neocortex, and the foundation of intelligence . . . Evolution discovers that if it tacks on a memory system (the neocortex) to the sensory path of the primitive brain, the animal gains an ability to predict the future . . . This new idea of the memory-prediction framework of the brain . . . “ (p. 84-105)

We can notice there is an intellectual drift in his reasoning. Evolution does not have a mind or intelligence. Just as we can prove human articulated language is the result of the conceptualizing power of the brain on one hand, and of other physical mutations dictated by the long distance bipedal nature of Homo Sapiens (not the first hominid to have that characteristic but the first to be endowed with mutations that go a lot farther than before) that are absolutely necessary for survival on the other hand (low larynx, high level of innervation of the laryngeal-glottal-buccal masticatory and articulatory apparatus, high level of coordination of various organs and functions), we have to consider evolution as being a blind and unguided process that selects haphazard mutations when they are propitious to bringing a higher survival potential to a given species. It is quite obvious that the development of the neocortex of mammals into human neocortex provided Homo Sapiens with a higher survival potential. In other words Hawkins suffers of some teleological bias which is a way to escape from asking who did it and hence a way to exclude the possible religious answer. But that is wrong. We don’t have to answer the question of where does the logic of evolution comes from because we cannot answer this question with any scientific final elaboration.

Then Hawkins moves to the second hierarchy, that of the neo-cortex structures. The neocortex is divided into columns that are perpendicular to the surface of it. It has six layers. The first layer has few cells that have myriads of small dendrites connected to their neighbors by synapses that can build and rebuild themselves. Then they have three axons, two horizontal and lateral in the first layer connecting this cell to distant other cells all over the brain on one side and on the other side, the famous spindle cells, and a third one going down into lower layers of the neocortex. When layers 1, 2, 3 are activated the activating pattern goes to layer 5 and then layer six. In layers 1, 2, and 3 the pattern is analyzed to be finally identified in layer 5. Then it is moved to layer 6 where a prediction might be performed about what may come next from this identified pattern. Then the transmission branches into part of it being sent to the Thalamus and then back to layer 1 as a feed back and part of it being send simultaneously to motor areas for processing. Layer 4 is the layer where a newly learned pattern, identified by the Hippocampus arrives to activate the column, that is to say layers 5 and 6 and beyond. This can be summarized in a triple hierarchy: the mind must first discriminate an element, then identify and eventually name that discriminated element, and finally classify ort conceptualize this identified and named element. This basic conceptualization that has to be constructed in a child through education, just the way it was constructed in Homo Sapiens through experience.

It is important then to cross this approach with a phylogenic and psychogenetic approach of language to understand how language was invented and how it is learned. That of course would require a lot of space and it is not here it can be presented. But let’s say that three hierarchies can be seen in language and all of them can only be understood as the crossing of the neocortical capabilities of Homo Sapiens on one side, and the highly frail state of Homo Sapiens or the highly dependent state of a human newborn on the other side. These hierarchies are that of the word: consonantal roots, isolating characters or themes, and conjugation-declension fronds giving the three (maybe four) vast phylogenic families of languages: consonantal Semitic languages, isolating Chinese, Tibeto-Burman and Khmero-Vietic languages, and agglutinative (the vast Turkic family from Turkish to Siouan) or synthetic-analytic languages (Indo-European and Indo Aryan languages).

The triple syntax of any language: Categorial syntax (discriminating nouns and verbs, spatial units and temporal units), Functional syntax (building the sentence on the pattern [AGENT (feed-ER) – RELATION (feed) – PATIENT (feed-EE) – THEME (feed-Ø, food, fodder)] and finally Expressive syntax (expressing the mood and modalizations imposed onto the utterance by the speaker and his relations to his environment. These three syntactic functions are mapped onto the first hierarchy by making it all discursive in root-languages, making the last two discursive in theme-languages and only keeping the expressive level for discursive means in frond-languages. Note each one of these three syntaxes is a hierarchy too by themselves.

Taking language into account would have enabled Hawkins to understand that he cannot consider the mind is the brain. The mind is an abstract and absolutely virtual construct of the brain from the various patterns the brain has registered in its own cells and molecules. I insist here on molecules because Microtubule Associated Proteins have been proved as having a role to play in various mental operation, particular with the loss of ,their phosphorylation when activated by some stimulus, for one example. The mind is based on the hierarchical potential and architecture of the brain and this potential and architecture produce the conceptualizing potential that will produce the virtual mind and its tools. These tools are essential if we want to understand the emergence of Homo Sapiens as the superior intelligent mammal on earth and if we want to understand today’s man and human society. The first of these tools is (spoken) language (note written language was invented only around 5 or 6 thousand years ago some 300,000 years after the invention of spoken language). Then Homo Sapiens invented all “ideological” tools to understand and explain the world in order to survive and expand in a state of great physical inferiority as compared to most of his predators. These tools are religion, astronomy, science, history, all constructed models of the world produced or that could be produced with the conceptualizing power of the human brain. Note here Neanderthals could not even invent fishing whereas Homo Sapiens just started with fishing to move onto agriculture, herd-husbandry, and so, and all that before inventing written language.

So I do not believe “the mind is just a label of what the brain does.” (p. 204) and the mind the way I have sketched it is something that might be one day equaled by machines. But these minded machines will not be human since they will not be able to learn and develop their brain and mind the way man does it, from scratch and as the result of an intense and highly emotional intercourse between an individual and his/her linguistic, cultural, social and emotional environment. We are not speaking of a machine loving a man, but of a machine loving a machine not as something programmed but as something learned from experience. As a matter of fact the Terminator saga is a lot more instructive on that point than what Hawkins says. In the same way the intelligent machines are not the machines themselves but all the Mr. Smith taking over the earth by decision of the Architect who manipulates machines into attacking humans till one, two or three humans are able to negotiate the end of the war with machines who accepts on the basis of Neo being crucified in order to be able to defeat all the Mr. Smith and the Architect’s matrix. Once again we are far away from what Hawkins says.

To conclude, Hawkins’s book is the first important step against the apocalyptic messianic prophetic prediction the engineers turned theoreticians like Ray Kurzweil who are already taking all the necessary pills to be able to live long and merge with intelligent machines in less than fifty years, and thus become the nurtured cows of these intelligent machines, who would not be intelligent enough to understand that kind of slavery would be doomed to destruction just like any other form of slavery was and has been doomed to destruction. If these machines were humanly intelligent they would understand that as a basic requirement to qualify for intelligence.

But at the same time Hawkins does not reach the level of the mind. He locks himself in the physiological and biological brain pretending it is the mind mixing up the capacity and the potential. He thinks too much with metaphors and comparisons. To use one I would say that a plane CAN fly but that this plane is not the FLYING POTENTIAL itself. The plane has that potential but to realize it a whole procedure is necessary (with kerosene, air strips, engineers in the air traffic control tower, pilots, passengers, freight, stewards and stewardesses, etc) and flying can only become a reality when that procedure has been performed. Hence the FLYING POTENTIAL is a VIRTUAL capability of the plane, just like the MIND is a virtual construct of the brain using its POTENTIAL INTELLIGENCE, and this POTENTIAL INTELLIGENCE cannot produce any INTELLIGENT ACTION if the VIRTUAL MIND is not activated and used by the brain.

The first intelligent machine invented by man was language in order to satisfy the need for communication Homo Sapiens had. That language has had a long career in improving and developing man’s lot. It has also transformed its inventor and his/her society.

There still is a long way to go to even approach such humanly intelligent machines. In the meantime we will invent and use more and more intelligent machines that will liberate our brain and body of innumerable tasks that would otherwise use our mental and physical time and energy. With this mental and physical time and energy we will develop new forms of intelligence that we cannot even imagine today, and we must not forget that evolution goes on and man is a natural species. The more contact he/she will have with intelligent machines the more chances there will be he/she will go through mutations and developments that will be retained by evolution and education as vastly increasing human intelligence. The more intelligent machines, the more chances man will become more intelligent.

Dr Jacques COULARDEAU