Imagine a band of ancestral hominids about to invent language.
Clearly, they did not begin by having a leader say, "Hey, look at
this--let's call it a banana. All of you say after me, ba-na-na."
Undoubtedly, though, the group had a set of capacities that prepared
the ground for systematic verbal communication. Our studies of the
neurobiological basis of synesthesia suggest that a facility for
metaphor--for seeing deep links between superficially dissimilar and
unrelated things--provided a key seed for the eventual emergence of
language.
Humans have a built-in bias to associate certain sounds with particular
visual shapes, which could well have been important in getting hominids
started on a shared vocabulary. In addition, specific brain areas that
process visual shapes of objects, letters and numbers, and word sounds
can activate each other even in nonsynesthetes, causing people to
expect, say, jagged shapes to have harsh-sounding names.
Two other types of neural connections support our idea. First, the
sensory areas for visual shapes and for hearing in the back of the brain
can cross-activate specific motor areas in the front of the brain that
participate in speech. A sharp visual inflection or a harsh sound
induces the motor control area for speech to produce an equally sudden
inflection of the tongue on the palate. (Or consider the spoken words
"diminutive," "teeny-weeny" and "un peu," which involve pursing
the lips to mimic the small size of the object.) The brain seems to
possess preexisting rules for translating what we see and hear into
mouth motions that reflect those inputs.
Second, a kind of spillover of signals occurs between two nearby motor
areas: those that control the sequence of muscle movements required for
hand gestures and those for the mouth. We call this effect "synkinesia."
As Charles Darwin pointed out, when we cut paper with scissors, our
jaws may clench and unclench unconsciously as if to echo the hand
movements. Many linguists do not like the theory that manual gesturing
could have set the stage for vocal language, but we believe that
synkinesia suggests that they may be wrong.
Assume that our ancestral hominids communicated mainly through emotional
grunts, groans, howls and shrieks, which are known to be produced by
the right hemisphere and an area in the frontal lobes concerned with
emotion. Later the hominids developed a rudimentary gestural system that
became gradually more elaborate and sophisticated; it is easy to
imagine how the hand movement for pulling someone toward you might have
progressed to a "come hither" wave. If such gestures were translated
through synkinesia into movements of the mouth and face muscles, and if
emotional guttural utterances were channeled through these mouth and
tongue movements, the result could have been the first spoken words.
How would we import syntax, the rules for using words and phrases in
language, into this scheme? We believe that the evolution of tool use by
hominids may have played an important role. For example, the
tool-building sequence--first shape the hammer's head, then attach it to
a handle, then chop the meat--resembles the embedding of clauses within
larger sentences. Following the lead of psychologist Patricia
Greenfield of the University of California at Los Angeles, we propose
that frontal brain areas that evolved for subassembly in tool use may
later have been co-opted for a completely novel function--joining words
into phrases and sentences.
Not every subtle feature of modern language is explained by such
schemes, but we suspect that these elements were critical for setting in
motion the events that culminated
in modern language.
Of the various manifestations of synesthesia, the most common involves seeing monochromatic letters, digits and words in unique colors—this is called grapheme-color synesthesia. For example, is both blue (real color) and light green (synesthetic color). Thus, synesthetic color differences can facilitate performance on tasks in which real color differences facilitate ...
Thomas J. Palmeri, Randolph B. Blake and René Marois of the psychology department and the Center for Integrative and Cognitive Neuroscience at Vanderbilt University
When you eat chicken, does it feel pointy or round? Is a week shaped like a tipped-over D with the days arranged counterclockwise? Does the note B taste like horseradish? Do you get confused about appointments because Tuesday and Thursday have the same color? Do you go to the wrong train station in New York City because Grand Central has the same color as the 42nd Street address of Penn Station? When you read a newspaper or listen to someone speaking do you see a rainbow of colors? If so, you might have synesthesia.
Synesthesia is an anomalous blending of the senses in which the stimulation of one modality simultaneously produces sensation in a different modality. Synesthetes hear colors, feel sounds and taste shapes. What makes synesthesia different from drug-induced hallucinations is that synesthetic sensations are highly consistent: for particular synesthetes, the note F is always a reddish shade of rust, a 3 is always pink or truck is always blue.
The estimated occurrence of synesthesia ranges from rarer than one in 20,000 to as prevalent as one in 200. Of the various manifestations of synesthesia, the most common involves seeing monochromatic letters, digits and words in unique colors—this is called grapheme-color synesthesia. One rather striking observation is that such synesthetes all seem to experience very different colors for the same graphemic cues. Different synesthetes may see 3 in yellow, pink or red. Such synesthetic colors are not elicited by meaning, because 2 may be orange but two is blue and 7 may be red but seven is green. Even more perplexing is that synesthetes typically report seeing both the color the character is printed in as well as their synesthetic color. For example, is both blue (real color) and light green (synesthetic color).
Synesthetes report having unusually good memory for things such as phone numbers, security codes and polysyllabic anatomical terminology because digits, letters and syllables take on such a unique panoply of colors. But synesthetes also report making computational errors because 6 and 8 have the same color and claim to prejudge couples they meet because the colors of their first names clash so hideously.
For too long, synesthetes were dismissed as having overactive imaginations, confusing memories for perceptions or taking metaphorical speech far too literally. Recent research, however, has documented the reality of synesthesia and is beginning to make headway into understanding what might cause such unusual perceptions.
Research has documented that synesthetic colors are perceived in much the same way that nonsynesthetic individuals perceive real colors. Thus, synesthetic color differences can facilitate performance on tasks in which real color differences facilitate performance for nonsynesthetes and can impair performance on tasks in which real color differences impair performance for nonsynesthetes.
In one such task, people are asked to say the color of the ink a word is printed in as quickly as possible (for example, responding "pink" to and "blue" to ). For lexical synesthetes, these words take on unique colors. When the synesthetic color matches the ink color, responses are fast. But when the synesthetic color mismatches the ink color, responses are slow, presumably because subjects need to resolve the conflict over which color name to respond with. Although such results demonstrate that synesthesia is automatic, in the sense that they cannot turn off their synesthesic experience even when it interferes with a task, these results do not reveal whether synesthetic colors are perceptions or memories.
To demonstrate the perceptual reality of synesthetic colors, researchers have introduced synesthetic color differences into a variety of traditional visual-perception tasks. Searching for a among ’s is a difficult task because the digits are so visually similar, differing by only a mirror reflection. If the was colored orange and the ’s were colored green, the search task would be trivially easy because the orange digit visually pops out from the background of green digits. When shown a display consisting of monochromatic digits, we found that a synesthete could quickly find the target because for him was orange but was green (see image).
Vilayanur Ramachandran and Edward M. Hubbard of the University of California at San Diego, have reported complementary findings supporting the perceptual reality of synesthetic colors. In one task, they presented synesthetes with an array of equally-spaced letters and digits. Synesthetes reported that these arrays organized themselves into distinct rows or columns depending on whether the rows or columns of characters were the same synesthetic color. This perceptual grouping based on synesthetic color is analogous to the kind of perceptual grouping non-synesthetes experience with real colors.
Ask the Experts: Biology 06/17/2002
Hearing Colors, Tasting Shapes Feature Article psychophysics, synaesthesia, synesthesia, neuroscience In the extraordinary world of synesthesia, senses mingle together--revealing some of the brain's mysteries People with synesthesia--whose senses blend together--are providing valuable clues to understanding the organization and functions of the human brain 15-APR-03 When Matthew Blakeslee shapes hamburger patties with his hands, he experiences a vivid bitter taste in his mouth. Blakeslee, ...
Hearing Colors, Tasting Shapes
People with synesthesia--whose senses blend together--are providing valuable clues to understanding the organization and functions of the human brain
By Vilayanur S. Ramachandran and Edward M. Hubbard
When Matthew Blakeslee shapes hamburger patties with his hands, he experiences a vivid bitter taste in his mouth. Esmerelda Jones (a pseudonym) sees blue when she listens to the note C sharp played on the piano; other notes evoke different hues--so much so that the piano keys are actually color-coded, making it easier for her to remember and play musical scales. And when Jeff Coleman looks at printed black numbers, he sees them in color, each a different hue. Blakeslee, Jones and Coleman are among a handful of otherwise normal people who have synesthesia. They experience the ordinary world in extraordinary ways and seem to inhabit a mysterious no-man's-land between fantasy and reality. For them the senses--touch, taste, hearing, vision and smell--get mixed up instead of remaining separate.
Modern scientists have known about synesthesia since 1880, when Francis Galton, a cousin of Charles Darwin, published a paper in Nature on the phenomenon. But most have brushed it aside as fakery, an artifact of drug use (LSD and mescaline can produce similar effects) or a mere curiosity. About four years ago, however, we and others began to uncover brain processes that could account for synesthesia. Along the way, we also found new clues to
Overview/Synesthesia
· Synesthesia (from the Greek roots syn, meaning "together," and aisthesis, or "perception") is a condition in which otherwise normal people experience the blending of two or more senses.
· For decades, the phenomenon was often written off as fakery or simply memories, but it has recently been shown to be real. Perhaps it occurs because of cross activation, in which two normally separate areas of the brain elicit activity in each other.
· As scientists explore the mechanisms involved in synesthesia, they are also learning about how the brain in general processes sensory information and uses it to make abstract connections between seemingly unrelated inputs.
|
some of the most mysterious aspects of the human mind, such as the emergence of abstract thought, metaphor and perhaps even language.
A common explanation of synesthesia is that the affected people are simply experiencing childhood memories and associations. Maybe a person had played with refrigerator magnets as a child and the number 5 was red and 6 was green. This theory does not answer why only some people retain such vivid sensory memories, however. You might think of cold when you look at a picture of an ice cube, but you probably do not feel cold, no matter how many encounters you may have had with ice and snow during your youth.
Another prevalent idea is that synesthetes are merely being metaphorical when they describe the note C flat as "red" or say that chicken tastes "pointy"--just as you and I might speak of a "loud" shirt or "sharp" cheddar cheese. Our ordinary language is replete with such sense-related metaphors, and perhaps synesthetes are just especially gifted in this regard.
Sidebar: Mingled Signals
|
We began trying to find out whether synesthesia is a genuine sensory experience in 1999. This deceptively simple question had plagued researchers in this field for decades. One natural approach is to start by asking the subjects outright: "Is this just a memory, or do you actually see the color as if it were right in front of you?" When we tried asking this question, we did not get very far. Some subjects did respond, "Oh, I see it perfectly clearly." But a more frequent reaction was, "I kind of see it, kind of don't" or "No, it is not like a memory. I see the number as being clearly red but I also know it isn't; it's black. So it must be a memory, I guess."
To determine whether an effect is truly perceptual, psychologists often use a simple test called pop-out or segregation. If you look at a set of tilted lines scattered amid a forest of vertical lines, the tilted lines stand out. Indeed, you can instantly segregate them from the background and group them mentally to form, for example, a separate triangular shape. Similarly, if most of a background's elements were green dots and you were told to look for red targets, the reds would pop out. On the other hand, a set of black 2's scattered among 5's of the same color almost blend in [see sidebar]. It is hard to discern the 2's without engaging in an item-by-item inspection of numbers, even though any individual number is just as clearly different from its neighbors as a tilted line is from a straight line. We thus may conclude that only certain primitive, or elementary, features, such as color and line orientation, can provide a basis for grouping. More complex perceptual tokens, such as numbers, cannot do so.
We wondered what would happen if we showed the mixed numbers to synesthetes who experience, for instance, red when they see a 5 and green with a 2. We arranged the 2's so that they formed a triangle. If synesthesia were a genuine sensory effect, our subjects should easily see the triangle because for them, the numbers would look colored.
When we conducted pop-out tests with volunteers, the answer was crystal clear. Unlike normal subjects, synesthetes correctly reported the shape formed by groups of numbers up to 90 percent of the time (exactly as nonsynesthetes do when the numbers actually have different colors). This result proves that the induced colors are genuinely sensory and that synesthetes are not just making things up. It is impossible for them to fake their success. In another striking example, we asked a synesthete who sees 5 tinged red to watch a computer display. He could not tell when we surreptitiously added an actual red hue to the white number unless the red was sufficiently intense; he could instantly spot a real green added to the 5.
Visual Processing
Confirmation that synesthesia is real brings up the question, Why do some people experience this weird phenomenon? Our experiments lead us to favor the idea that synesthetes are experiencing the result of some kind of cross wiring in the brain. This basic concept was initially proposed about 100 years ago, but we have now identified where in the brain and how such cross wiring might occur.
An understanding of the neurobiological factors at work requires some familiarity with how the brain processes visual information [see illustration on opposite page]. After light reflected from a scene hits the cones (color receptors) in the eye, neural signals from the retina travel to area 17, in the occipital lobe at the back of the brain. There the image is processed further within local clusters, or blobs, into such simple attributes as color, motion, form and depth. Afterward, information about these separate features is sent forward and distributed to several far-flung regions in the temporal and parietal lobes. In the case of color, the information goes to area V4 in the fusiform gyrus of the temporal lobe. From there it travels to areas that lie farther up in the hierarchy of color centers, including a region near a patch of cortex called the TPO (for the junction of the temporal, parietal and occipital lobes). These higher areas may be concerned with more sophisticated aspects of color processing. For example, leaves look as green at dusk as they do at midday, even though the mix of wavelengths reflected from the leaves is very different.
Sidebar: Color-Coded World
|
Numerical computation, too, seems to happen in stages. An early step also takes place in the fusiform gyrus, where the actual shapes of numbers are represented, and a later one occurs in the angular gyrus, a part of the TPO that is concerned with numerical concepts such as ordinality (sequence) and cardinality (quantity). (When the angular gyrus is damaged by a stroke or a tumor, the patient can still identify numbers but can no longer divide or subtract. Multiplication often survives because it is learned by rote.) In addition, brain-imaging studies in humans strongly hint that visually presented letters of the alphabet or numbers (graphemes) activate cells in the fusiform gyrus, whereas the sounds of the syllables (phonemes) are processed higher up, once again in the general vicinity of the TPO.
Because both colors and numbers are processed initially in the fusiform gyrus and subsequently near the angular gyrus, we suspected that number-color synesthesia might be caused by cross wiring between V4 and the number-appearance area (both within the fusiform) or between the higher color area and the number-concept area (both in the TPO). Other, more exotic forms of the condition might result from similar cross wiring of different sensory-processing regions. That the hearing center in the temporal lobes is also close to the higher brain area that receives color signals from V4 could explain sound-color synesthesia. Similarly, Matthew Blakeslee's tasting of touch might occur because of cross wiring between the taste cortex in a region called the insula and an adjacent cortex representing touch by the hands.
Sidebar: The Puzzle of Language
|
Assuming that neural cross wiring does lie at the root of synesthesia, why does it happen? We know that it runs in families, so it has a genetic component. Perhaps a mutation causes connections to emerge between brain areas that are usually segregated. Or maybe the mutation leads to defective pruning of preexisting connections between areas that are normally connected only sparsely. If the mutation were to be expressed (that is, to exert its effects) in some brain areas but not others, this patchiness might explain why some synesthetes conflate colors and numbers whereas others see colors when they hear phonemes or musical notes. People who have one type of synesthesia are more likely to have another, which adds weight to this idea.
Although we initially thought in terms of physical cross wiring, we have come to realize that the same effect could occur if the wiring--the number of connections between regions--was fine but the balance of chemicals traveling between regions was skewed. So we now speak in terms of cross activation. For instance, neighboring brain regions often inhibit one another's activity, which serves to minimize cross talk. A chemical imbalance of some kind that reduces such inhibition--for example, by blocking the action of an inhibitory neurotransmitter or failing to produce an inhibitor--would also cause activity in one area to elicit activity in a neighbor. Such cross activation could, in theory, also occur between widely separated areas, which would account for some of the less common forms of synesthesia.
Support for cross activation comes from other experiments, some of which also help to explain the varied forms synesthesia can take. One takes advantage of a visual phenomenon known as crowding [see sidebar]. If you stare at a small plus sign in an image that also has a number 5 off to one side, you will find that it is easy to discern that number, even though you are not looking at it directly. But if we now surround the 5 with four other numbers, such as 3's, then you can no longer identify it. It looks out of focus. Volunteers who perceive normally are no more successful at identifying this number than mere chance. That is not because things get fuzzy in the periphery of vision. After all, you could see the 5 perfectly clearly when it wasn't surrounded by 3's. You cannot identify it now because of limited attentional resources. The flanking 3's somehow distract your attention away from the central 5 and prevent you from seeing it.
A broadcast version of this article will air April 24 on National Geographic Today, a program on the National Geographic Channel. Please check your local listings.
|
A big surprise came when we gave the same test to two synesthetes. They looked at the display and made remarks like, "I cannot see the middle number. It's fuzzy but it looks red, so I guess it must be a 5." Even though the middle number did not consciously register, it seems that the brain was nonetheless processing it somewhere. Synesthetes could then use this color to deduce intellectually what the number was. If our theory is right, this finding implies that the number is processed in the fusiform gyrus and evokes the appropriate color before the stage at which the crowding effect occurs in the brain; paradoxically, the result is that even an "invisible" number can produce synesthesia.
Another finding we made also supports this conclusion. When we reduced the contrast between the number and the background, the synesthetic color became weaker until, at low contrast, subjects saw no color at all, even though the number was perfectly visible. Whereas the crowding experiment shows that an invisible number can elicit color, the contrast experiment conversely indicates that viewing a number does not guarantee seeing a color. Perhaps low-contrast numbers activate cells in the fusiform adequately for conscious perception of the number but not enough to cross-activate the color cells in V4.
Finally, we found that if we showed synesthetes Roman numerals, a V, say, they saw no color--which suggests that it is not the numerical concept of a number, in this case 5, but the grapheme's visual appearance that drives the color. This observation, too, implicates cross activation within the fusiform gyrus itself in number-color synesthesia, because that structure is mainly involved in analyzing the visual shape, not the high-level meaning of the number. One intriguing twist: Imagine an image with a large 5 made up of little 3's; you can see either the "forest" (the 5) or focus minutely on the "trees" (the 3's). Two synesthete subjects reported that they saw the color switch, depending on their focus. This test implies that even though synesthesia can arise as a result of the visual appearance alone--not the high-level concept--the manner in which the visual input is categorized, based on attention, is also critical.
But as we began to recruit other volunteers, it soon became obvious that not all synesthetes who colorize their world are alike. In some, even days of the week or months of the year elicit colors. Monday might be green, Wednesday pink, and December yellow.
The only thing that days of the week, months and numbers have in common is the concept of numerical sequence, or ordinality. For certain synesthetes, perhaps it is the abstract concept of numerical sequence that drives the color, rather than the visual appearance of the number. Could it be that in these individuals, the cross wiring occurs between the angular gyrus and the higher color area near the TPO instead of between areas in the fusiform? If so, that interaction would explain why even abstract number representations, or the idea of the numbers elicited by days of the week or months, will strongly evoke specific colors. In other words, depending on where in the brain the mutant gene is expressed, it can result in different types of the condition--"higher" synesthesia, driven by numerical concept, or "lower" synesthesia, produced by visual appearance alone. Similarly, in some lower forms, the visual appearance of a letter might generate color, whereas in higher forms it is the sound, or phoneme, summoned by that letter; phonemes are represented near the TPO.
We also observed one case in which we believe cross activation enables a colorblind synesthete to see numbers tinged with hues he otherwise cannot perceive; charmingly, he refers to these as "Martian colors." Although his retinal color receptors cannot process certain wavelengths, we suggest that his brain color area is working just fine and being cross-activated when he sees numbers.
In brain-imaging experiments we are conducting with Geoff Boynton of the Salk Institute for Biological Studies in San Diego, we have obtained preliminary evidence of local activation of the color area V4 in a manner predicted by our cross-activation theory of synesthesia. (Jeffrey Gray of the Institute of Psychiatry in London and his colleagues have reported similar results.) On presenting black and white numbers to synesthetes, brain activation arose not only in the number area--as it would in normal subjects--but also in the color area. Our group also observed differences between types of synesthetes. One of our subjects with lower synesthesia showed much greater activation in earlier stages of color processing than occurred in controls. In contrast, higher synesthetes show less activation at these earlier levels.
A Way with Metaphor
Our insights into the neurological basis of synesthesia could help explain some of the creativity of painters, poets and novelists. According to one study, the condition is seven times as common in creative people as in the general population.
One skill that many creative people share is a facility for using metaphor ("It is the east, and Juliet is the sun"). It is as if their brains are set up to make links between seemingly unrelated domains--such as the sun and a beautiful young woman. In other words, just as synesthesia involves making arbitrary links between seemingly unrelated perceptual entities such as colors and numbers, metaphor involves making links between seemingly unrelated conceptual realms. Perhaps this is not just a coincidence.
Numerous high-level concepts are probably anchored in specific brain regions, or maps. If you think about it, there is nothing more abstract than a number, and yet it is represented, as we have seen, in a relatively small brain region, the angular gyrus. Let us say that the mutation we believe brings about synesthesia causes excess communication among different brain maps--small patches of cortex that represent specific perceptual entities, such as sharpness or curviness of shapes or, in the case of color maps, hues. Depending on where and how widely in the brain the trait was expressed, it could lead to both synesthesia and to a propensity toward linking seemingly unrelated concepts and ideas--in short, creativity. This would explain why the apparently useless synesthesia gene has survived in the population.
In addition to clarifying why artists might be prone to experiencing synesthesia, our research suggests that we all have some capacity for it and that this trait may have set the stage for the evolution of abstraction--an ability at which humans excel. The TPO (and the angular gyrus within it), which plays a part in the condition, is normally involved in cross-modal synthesis. It is the brain region where information from touch, hearing and vision is thought to flow together to enable the construction of high-level perceptions. For example, a cat is fluffy (touch), it meows and purrs (hearing), it has a certain appearance (vision) and odor (smell), all of which are derived simultaneously by the memory of a cat or the sound of the word "cat."
Could it be that the angular gyrus--which is disproportionately larger in humans compared with that in apes and monkeys--evolved originally for cross-modal associations but then became co-opted for other, more abstract functions such as metaphors? Consider two drawings, originally designed by psychologist Wolfgang Köhler. One looks like an inkblot and the other, a jagged piece of shattered glass. When we ask, "Which of these is a 'bouba,' and which is a 'kiki'?" 98 percent of people pick the inkblot as a bouba and the other one as a kiki. Perhaps that is because the gentle curves of the amoebalike figure metaphorically mimic the gentle undulations of the sound "bouba" as represented in the hearing centers in the brain as well as the gradual inflection of the lips as they produce the curved "boo-baa" sound. In contrast, the waveform of the sound "kiki" and the sharp inflection of the tongue on the palate mimic the sudden changes in the jagged visual shape. The only thing these two kiki features have in common is the abstract property of jaggedness that is extracted somewhere in the vicinity of the TPO, probably in the angular gyrus. (We recently found that people with damage to the angular gyrus lose the bouba-kiki effect--they cannot match the shape with the correct sound.) In a sense, perhaps we are all closet synesthetes.
So the angular gyrus performs a very elementary type of abstraction--extracting the common denominator from a set of strikingly dissimilar entities. We do not know how exactly it does this job. But once the ability to engage in cross-modal abstraction emerged, it might have paved the way for the more complex types of abstraction. The opportunistic takeover of one function for a different one is common in evolution. For example, bones in the ear used for hearing in mammals evolved from the back of the jawbone in reptiles. Beyond metaphor and abstract thinking, cross-modal abstraction might even have provided seeds for language [see sidebar].
When we began our research on synesthesia, we had no inkling of where it would take us. Little did we suspect that this eerie phenomenon, long regarded as a mere curiosity, might offer a window into the nature of thought.
VILAYANUR S. RAMACHANDRAN AND EDWARD M. HUBBARD collaborate on studies of synesthesia. Ramachandran directs the Center for Brain and Cognition at the University of California San Diego Trinity College University of Cambridge All Souls College University of Oxford Royal Netherlands Academy American Academy
1bio
1bio
