What are mental codes?

Consider what has to go on in your mind in order for you to carry out the instructions for a typical Choice Reaction Time task, such as: “press the left key when you see the color green and the right key when you see the color blue.”

When a stimulus appears, at least three things have to happen: 1) you have to figure out the color of the stimulus, 2) you have to decide which key to press, and 3) you have to actually press the key. These are usually considered the three most basic, broadly-defined processes involved in carrying out a task, and are usually called stimulus identification, response selection, and motor programming, respectively (although they can be broken down into more specific sub-processes, as well; see Sanders, 1980, 1990).

These processes can be described more concretely in terms of information and mental codes. Your senses give you signals that contain information about what is going on in the world around you. In order to understand and react to the world, you use that information to create a mental picture of your environment. In other words, you form a stimulus code: a mental representation of what properties are in the stimulus environment that produced the sensory signals that you received.

Stimulus identification can be thought of as the process of forming stimulus codes based on sensory information. Those stimulus codes, in turn, contain information that can be used to decide on a response. In order to act on the world, you form a response code: a mental representation of what actions you want to carry out. Response selection can be thought of as the process of forming response codes based on information from stimulus codes. Finally, motor programming is the process of using information from response codes t prepare specific muscular movements that carry out your response. This can be thought of as the formation of motor codes, which are the programs your muscles use to make a response.

The idea that thought and action in the world consists of mental codes (representations of stimulus properties and response actions) is called the information processing approach, and models of performance based on this framework are called information processing models (see Anderson 1995; Bower 1975; Miller, 1988). Performing a task requires transforming information from the world into a stimulus code, a response code, and then a motor code, through a sequence of mental processes.

These processes clearly depend on one another. In the example above, what key you press depends on what side (left or right) you decide is correct, and what side you decide is correct depends on what you think the color of the stimulus is. In the language of information processing models, the output of stimulus identification, which contains information about the stimulus code, is used as the input for response selection. Similarly, the output of response selection, which contains information about the response code, is used as input for motor execution. Information processing models use terms like “input” and “output” a lot, because they were originally motivated by the idea that mental processes are like computer programs, and mental codes are like computer data (see Newell, Rosenbleem, & Laird, 1989; Simon, 1981; Simon & Kaplan, 1989).

Psychologists want to know exactly what is going on in these processes; that is, how information is represented in these codes, and how they are actually formed. One way to approach this question is to measure people’s performance, their speed and accuracy when carrying out a task, under different kinds of task conditions. The amount of time it takes for you to make a response is related to how difficult each of these processes is: when something about the task makes your response faster or slower, it is because one (or more) of these processes has been helped or hindered. By examining how different kinds of task conditions influence performance, psychologists are able to get an idea about what is actually going on in the formation of these different mental codes. This approach is called mental chronometry (see Meyer et al., 1988; Sanders, 1993).

There are a number of specific questions one can ask about the formation of mental codes during choice reaction time tasks. Is input information compared to items in memory one by one, until a match is found? Or is the input information compared to all possible items in memory at once? Does input information for a process cause mental codes to form gradually, or do mental codes form in chunks, like “yes” and “no” decisions? Does incomplete information get used by later processes, or do they have to wait until the previous process is completed?

Even more questions can be asked about consistency effects in classification tasks. How does irrelevant information affect the formation of mental codes? Does it influence the formation of stimulus codes, response codes, or motor codes? Does irrelevant information always have the same kind of influence on mental codes, or does it depend on task conditions?

Today, most models of consistency effects share a few basic assumptions about mental codes and how they behave during classification tasks (see, e.g., Barber & O’Leary, 1997; Kornblum et al., 1990; O’Leary & Barber, 1993; Lu & Procter, 1995; Prinz, 1990; Proctor, Reeve, & van Zandt, 1992; Umilta & Nicoletti, 1990; Wallace, 1971). For example, they assume that irrelevant stimulus codes form automatically, that different stimulus features are formed by multiple parallel identification processes, that mental codes are abstract representations, and that mental codes form gradually over time.

However, they also disagree on a few very key assumptions about mental processing. For example, different models often disagree about where selective inhibition happens. They also can disagree about whether the formation of response codes from stimulus information is continuous or happens only in discrete stage-like chunks. Finally, they can disagree about whether irrelevant stimulus information influences the formation of stimulus codes, the formation of response codes, or both.

This last question is the key issue that differentiates the Dimensional Overlap Model from other models of consistency effects. Most models of consistency effects assume that irrelevant stimulus information influences the formation of response codes, whereas the Dimensional Overlap Model assumes that influence of the irrelevant stimulus depends on what the irrelevant stimulus dimension overlaps with.

 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn

Where does selective inhibition happen?

One question about which there is little consensus among connectionist models of reaction time, is the question of the locus of inhibition between alternatives.  Are inhibitory mechanisms active within each cognitive process, or between them?

According to the within-process inhibition view, the activation of one mental code in a particular process (i.e. within a module) will inhibit all of the other mental codes in that process (i.e. within the same module).  According to the between-process inhibition view, activation of a mental code in one process (e.g. stimulus identification) will directly inhibit non-corresponding codes in the following process (e.g. response selection).

Most of the earliest connectionist network models, including McClelland’s (1979) Cascade model, were feed-forward networks (see Rumelhart, McCelland, et al., 1986).  This meant that any given unit could only supply input to units in later processes, and there were no connections at all between units within the same module.  As a result, these network models naturally had to implement between-process inhibition: each stimulus unit had a positive association with one or more response units, and negative associations with all of the rest of the response units.

This feed-forward architecture also arises from many “learning algorithms”: processes through which the association strengths in a network can change from trial to trial, “learning” based on experience.  For example, the model of the Stroop effect by Cohen, Dunbar and McClelland (1990) assigns association strengths based on a standard learning algorithm (known as backpropagation) that “trains” the network on the process of reading (giving color word responses to word inputs) and on the process of color-naming (giving color word responses to color inputs).  The result of this algorithm is that, for example, the stimulus unit for the word “red” has a positive association with the response unit for the word “red,” and also a negative association with the response unit for the word “green”.  Similarly, the stimulus unit for the word “green” has a positive association with the response unit for the word “green”, and also a negative association with the response unit for the word “red”.  The result is between-process inhibition: negative associations connect stimulus units to response units.

McClelland and Rumelhart (1981), however, introduced a framework with a different assumption about connections between units.  They describe a semantic activation model with a feature module (with units representing mental codes for individual visual features), a letter module (with units representing mental codes for individual letters), and a word module (with units representing mental codes for whole words). They suggest that units are connected with positive or negative associations based on their consistency: that is, because the existence of the word “THE” is consistent with the existence of the letter “T” in the initial position, the initial “T” letter unit and the “THE” word unit have a positive association; on the other hand, because the existence of the word “ARE” is inconsistent with the existence of the letter “T” in the initial position, the initial “T” letter unit and the “ARE” word unit have a negative association.

One result of this kind of assumption is that all of the units within the same module are mutually inhibitory: the existence of a letter “T” in the initial position of a word is inconsistent with there being any other letter in that initial position, the existence of the word “ARE” is inconsistent with there being any other word, and so on.  As a result, this model implements within-process inhibition: activation of any unit within a module (process) inhibits activation of all of the other units in the same module.  However, this model also implements between-process inhibition, because the same rules apply to connections between units in different modules (e.g. letter units and word units).

This set of assumptions is used as the basis for the Stroop model developed by Phaf, van der Heijden, and Hudson (1990).  In this model, color units (representing alternative possible color codes) inhibited both each other (within-process inhibition) and inconsistent color word responses units (between-process inhibition).  Phaf et al. (1990) further argue in support of within-process inhibition by citing neurophysiological evidence: they claim that the neurophysiological phenomenon of “lateral inhibition” (inhibition among all of the nearby neurons in the same layer in cortical tissue) can be thought of as evidence that the brain implements within-process inhibition.

It was soon noted, of course, that having both of these kinds of inhibition is computationally redundant.  Several years later, McClelland (1993) proposed a normative framework for connectionist network models of performance called the Graded Random And Interactive Network (GRAIN) framework, in which he suggested that models should use only inhibitory connections between units in the same module, and only excitatory connections between units in different modules.  This effectively called for models of performance to constrain themselves to implementing within-module inhibition.  Most of the motivation for this move was computational, not psychological, and almost every “advantage” described by McClelland (1993, pp. 659-660) for within-process inhibition could also be found in an appropriately structured model with between-process inhibition.

Many models of consistency effects, nonetheless, have followed this normative suggestion (e.g. Barber & O’Leary, 1997; Cohen & Huston, 1994; Cohen, Servan-Schreiber, & McClelland, 1992; O’Leary & Barber, 1993; Zhang & Kornblum, 1998; Zhang, Zhang, & Kornblum, 1999; Zorzi & Umilta, 1995).  Zorzi and Umilta (1995), however, did explore the performance of an alternative version of their model that implemented between-process inhibition.  They found that both models could account for performance equally well, and that in the end the only motivation for preferring within-process inhibition was theoretical consistency: because everyone else was doing it.

Kornblum et al. (1999) pointed out that having within-process inhibition can lead to explosive inhibitory feedback effects if activation values are allowed to go below zero (see Kornblum et al., 1999,  p. 706).  Zorzi and Umilta (1995) and Cohen and Huston (1994) dealt with this by constraining the output of units between 0 and 1.  Instead of adding this additional constraint on the dynamics of processing, Kornblum et al. (1999) implement between-process inhibition.  By using this alternative inhibition mechanism, and forcing fewer constraints on processing (because output could be either positive or negative with no ill consequences to the model), they were still easily able to account for performance in Simon tasks, Stroop-like tasks, and their factorial combinations.


 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn