Photos Spare Cycles MythBusters

Talk: A Theory of Neocortex

Amazon Image

A Theory of Neocortex and its Implications for Machine Intelligence Jeff Hawkins Founder Palm Computing, Handspring Director, Redwood Neurosciences Institute Author of On Intelligence

First interest was studying brains, but hard to do in practice, so went to second interest, computing

Machine intelligence paradigms

1) AI (1940s-1980s) * computer programs * ignored biology, emulate human behavior (computer is general purpose)

2) Neural Nets (1970s-90s) * networks of neurons * classify spatial patterns * mostly ignore biology (don't try to capture structure of brain)

Neither of these approaches really "worked" in building intelligent machines

3) "Real Intelligence" (2005-). * biologically derived * hierarchical memory (brain is a type of memory system) * goal is pattern prediction (predict the future)

Introduction to the Neocortex

  • covers the rest of the brain
  • used sheet as "model" prop. In reality wrinkled because can't fit, but really is a sheet of cells
  • 30 trillion synapses
  • all high-level intelligence comes from neocortex (vision/auditory/hearing/motor/speaking/math/language/art/planning)

Neocortex divided into functional regions, all humans have fairly similar breakdown * If you look at the detailed structure of neocortex, no matter where you look it's pretty identical. Can't really tell the vision part from the auditory part. * 6 layers, different cell types in each layer * works same way everywhere (same mechanism for vision/motor/hearing, etc...). Single algorithm to do everything => If you can figure out how one part works, can apply elsewhere * high platicity of neural cortex

What does the Neocortex do?

1) the neocortex is a memory system (exposed to patterns, recalls them). Does not work like computer memory. 2) Through exposure, it builds a model of the world. 3) The neocortical memory model predicts future events by analogy to past events. Do this constantly in low and high level cognition.

Reptilian brain

  • Sophisticated senses -> Reptilian brain -> Behavior
  • no neocortex
  • sophisticated senses
  • sophisticted behavior: can run, hide
  • to a rough approximation, human brain is reptilian brain with neocortex on top

Mammalian brain

  • Sophisticated senses -> Reptilian brain -> Neocortex (records sense, reports expectation) -> Behavior

Human brain

  • large neocortex (not unique, but important)
  • unique in that it has a close relationship with motor behavior. uses existing prediction behavior. allows humans to generate long term planned behavior (e.g. speech)
  • hierarchical connectivity, can separate out into different regions
  • closer to lower levels: spatially specific (specific visual area, specific frequency), fast changing (during eye saccades see completely different patterns), "features"/"details" (specific line feature detectors)
  • closer to higher levels: spatially invariant (anything in visual area, any sound), slow changing (during eye saccades pattern stays same), "objects" (e.g. "face")
  • different levels communicate back and forth
  • can predict across senses (e.g. clapping hands -- visual + auditory)
  • all parts of cortex are involved in motor behavior

What does each region do?

Every region: 1. stores sequences (e.g. melody -- have to recall over time, not all at once) 2. passes sequence "name" up 3. predicts next element 4. converts invariant prediction into specific prediction (learn melody independent of actual notes, just relative intervals. As you pass prediction down melody prediction get converted into specific note, and vice versa) 5. passes specific prediction "down"

Memory efficient because break down knowledge into hierarchical structures within structures

Unanticipated events rise up in the hierarchy until some region can interpret it * if you have something truly novel, goes all the way to the top * when driving home, top levels of cortex free for other tasks because lower levels can handle task. If pig runs across the road, pops up to the top. * Hippocampus is at the top (hippocampus responsible for recording new memories, but memories do not reside there). Hippocampus passes novels events back down later to be remembered.

How does a region work - biology

Vertically connected through six layers of neocortex

Learning sequences * a little to complex to take notes on

Can we express this mathematically?


All inputs and outputs are probability distributions (two inputs [higher level, lower level], two outputs [higher level, lower level])


S = sequence

X = input

C = causes or context

C (Higher region input)



Sa(xt, xt+1, ...)

Sb(xt, xt+1, ...)


X (lower region input)

Recognition without context

X -> P(C)

Recognition with context can lead to new interpretation

X + C1 -> C1

Passing a belief down the hierarchy

(missed notes)

Predicting the future

Xt + C -> C + f(Xt+1, P(S|C))

Belief propagation can determine most likely causes of input in a hierarchy of conditional probabilities

Picture recognition problem

  • Matlab simulation
  • ~90 line-based symbols (letters, other madeup symbols)
  • When training system, have to move it around. Time is important (see it in different aspects)
  • Distributed representation (does not store template of symbol anywhere)
  • Does reasonable job of recognition
  • Prediction fill-in: If you take "dog" symbol and remove legs, will recognize dogs, and also predict that there should be legs. If you remove legs and head, incorrectly recognized as "cat" symbol and filled in different details (no legs, but head)

How do we build it/What problems can be solved?

  • believes model he presented is pretty close to actual workings of human brain
  • believes that it can solve current problems in field of AI, machine vision
  • eventually may want to build specific chips for this
  • don't think it's possible to use these techniques to pass a Turing test, believes would have to emulate full human (e.g. motor components) to pass
  • can probably build machine that thinks faster than human, more memory than human, but that's not the most interesting problem
  • most interesting thing is to apply this technique to alien problems to humans, e.g. weather prediction. should be able to build machines really smart at math and physics, help us understand world/cosmology


Will a computer be able to manipulate frontal lobes? no, too hard (small cells, complex). doesn't believe there will be these brain mind-meld devices. not in the near future

My thoughts

There might be some interesting parallels between this and Wolfram's work

I had heard about the whole prediction aspect of cognition, but when it was explained to me in my freshman seminar, it was from the viewpoint of "this is how the brain deals with the fact that your sensory input takes too long to reach the brain and be processed, i.e. you have to predict your sensory input in order to react quickly." Hawkin's perspective derived the prediction aspect from the brain's architecture itself, as a function of the bidirectionality of the hierarchical decomposition.

I thought the most interesting aspect was the emphasis on the bidirectionality in the architecture, the ability to recognize and predict simultaneously, as well as the ability of the brain to use the prediction mechanisms to keep regular occurences at lower levels.

Post a comment


related entries.

what is this?

This page contains a single entry from kwc blog posted on January 13, 2005 5:52 PM.

The previous post was Uncle kwc.

The next post is Book: A Partly Cloudly Patriot.

Current entries can be found on the main page.