Source: The Wall Street Journal - OCT 23, 2010
In a garage in Palo Alto, Calif., in the 1980s, some friends and I were the first humans to experience becoming avatars—that is, movable representations of ourselves in cyberspace. Amazingly, all these years later, almost no one else has been able to experience a hint of what will be one of the great cognitive adventures of this century.
It has been possible for some years for visitors to theme parks to try out virtual-reality "rides," but these don't capture the experience. Becoming an avatar in virtual reality, as a full-bodied human (or even nonhuman), has the potential to be vastly more interesting and important than one would expect from a technological amusement. What is really going on is the opening up of a new frontier of human potential, which can be called "somatic cognition"—somatic meaning "of the body."
I first became aware of somatic cognition while learning to improvise music at a piano. After enough practice, a moment comes when you notice that your hands have solved complicated puzzles of voice and harmony faster than your conscious mind can keep up. Fine basketball players, surgeons and pilots report similar moments.
In such cases, the human body is extended by physical objects that map body motion into a theater of thought and strategy not usually available to us. Objects like a basketball, a jet or a piano play a role in somatic cognition that is similar to the role of words in conscious thought.
The first hint that avatars could become the ultimate somatic objects came to me in the form of a software bug in that Palo Alto garage. I was inhabiting a humanoid avatar, but my hand was gigantic by accident. I bent my fingers and was suddenly engulfed by my fingertip.
The surprise was that I could still operate a huge hand effectively. This led to a question: How strange could an avatar get before a person could no longer control it?
The answer turns out to be an even bigger surprise. People can inhabit awfully odd avatars. One of the early avatars was a lobster—a creature with more limbs than a human. By mapping values from body poses, it turns out people can learn to inhabit other bodies not just with oddly shaped limbs, or limbs attached in unfamiliar places, but even bodies with different numbers of limbs.
This phenomenon is called "homuncular flexibility." The homunculus is the mapping of the body into the motor cortex, which is a portion of the brain located approximately under the portion of the scalp that would be occupied by a Mohawk hairdo.
That the mapping of the homunculus could be so flexible as to adapt to non-human bodies was initially a shock, but a delightful one. The sensation of inhabiting a nonhuman avatar is a new kind of pleasure. Think about what it would be like to wear wonderful clothing, combined with driving a superb vehicle, combined with mastering an extraordinary physical skill. It is like all those things together, but more expressive.
When we can successfully inhabit a nonhuman avatar, we are exploring not only the brain's deep history, but also the potential far future of all the creatures for which it is preadapted—what might happen in hundreds of millions of years. Becoming an avatar is a form of extreme time travel for the brain.
My favorite experiment so far involved turning elementary-school kids into the things they were studying.
Some were turned into molecules, dancing and squirming to dock with other molecules. In this case the molecule serves the role of the piano, and instead of harmony puzzles, you are learning chemistry. Somatic cognition offers an overwhelming emotional appeal for education, because it leverages vanity. You become the thing you are studying. Your sensory motor loop is modified to incorporate the logic of a science, and you develop body intuition about that logic.
No one knows how big a deal avatar-directed cognition will be. Will students routinely dance to learn chemistry in the future? Quite possibly. A student might also become a triangle to learn trigonometry, or a strand of DNA to learn about biology. Will professional nanotechnology engineers become molecular structures in order to refine them? Once again, it seems quite possible.