document: Fsdchap5
draft: 10/12/98
CHAPTER 5
EXPERIMENTAL EVIDENCE
I. Introduction. . . . . . . . . . . . . . . . . . . . . . .5-1
II. What Kind of Phenomena are to be Explained?. . . . . . . .5-5
III. Influential Studies . . . . . . . . . . . . . . . . . . .5-7
Mental Rotation . . . . . . . . . . . . . . . . . . . . .5-8
Shepard and Metzler. . . . . . . . . . . . . . . . .5-8
Learning and Mental Rotation . . . . . . . . . . . 5-10
Mental Scanning . . . . . . . . . . . . . . . . . . . . 5-15
Kosslyn. . . . . . . . . . . . . . . . . . . . . . 5-15
Pylyshyn's Objections to Scanning Experiments. . . 5-18
Finke. . . . . . . . . . . . . . . . . . . . . . . 5-19
Conclusion. . . . . . . . . . . . . . . . . . . . . . . 5-24
Problems for Empirical Cognitive Psychology. . . . . . . . . 5-25
Individual Variation. . . . . . . . . . . . . . . . . . 5-25
Underdetermination and Lack of Predictive Power . . . . 5-27
The Will and Conscious Judgment . . . . . . . . . . . . 5-34
Conclusion. . . . . . . . . . . . . . . . . . . . . . . 5-36
V. An Alternate Experimental Model: Chambers and Reisberg. . 5-37
Experiments . . . . . . . . . . . . . . . . . . . . . . 5-37
Qualifications of Results . . . . . . . . . . . . . . . 5-38
Conclusion. . . . . . . . . . . . . . . . . . . . . . . 5-42
VI. Concluding Remarks . . . . . . . . . . . . . . . . . . . 5-46
Theory and Experiment . . . . . . . . . . . . . . . . . 5-46
Epiphenomenalism. . . . . . . . . . . . . . . . . . . . 5-48
Computer User Dualism . . . . . . . . . . . . . . . . . 5-55
Figure 5-1 . . . . . . . . . . . . . . . . . . . . . . . . . 5-60
Figure 5-2 . . . . . . . . . . . . . . . . . . . . . . . . . 5-61
Figure 5-3 . . . . . . . . . . . . . . . . . . . . . . . . . 5-62
Figure 5-4 . . . . . . . . . . . . . . . . . . . . . . . . . 5-63
Figure 5-5 . . . . . . . . . . . . . . . . . . . . . . . . . 5-64
Figure 5-6 . . . . . . . . . . . . . . . . . . . . . . . . . 5-64
CHAPTER 5
EXPERIMENTAL EVIDENCE
I. Introduction
The contemporary debate was initiated primarily by concerns
about how to interpret experiments. Both sides agreed that
internal representations (structures or data) and cognitive
processes operating on them were needed to explain the
experimental data, but they disagreed about which structures and
processes were involved. Most of all, they disagreed about the
specific interpretation of several crucial experiments that were
thought to reveal imagery characteristics as inextricable from
the structure/process pair constituting the cognitive operation.
That is, they disagreed over the answer to the question we posed
at the beginning of Chapter 4:
Are there any root phenomena in the functional architecture
of the brain that are intrinsically pictorial or imagistic
in character?
As Kosslyn thought it was a theoretical possibility that imagery
data structures could exist in the brain, the only remaining
problem, from his point of view, was to find unequivocal
experimental evidence in support of it. This would complete his
parallelism hypothesis of a match in form between inner brain
representations and the form of our experience.
Pylyshyn's concern with empirical evidence was somewhat
different, since he posited no necessary relation between
cognitive processes and the form of our experiences. His concern
was to find phenomena that are cognitively impenetrable, that is,
phenomena that do not admit of being altered by the subject
through the application of knowledge about other things. The
infiltration of tacit knowledge, say, about the rules of objects
moving through space in the real world, would show that the
subject is not treating an internal representation as a given,
unalterable unit in processing, but as a unit that can be
creatively altered to fit the demands of the moment.
Cognitive science, in Pylyshyn's view, is concerned to
distinguish cognitive phenomena, or phenomena that are governed
by the application of knowledge (the use rules and
representations), from those that are non-cognitive, root
phenomena, or what Pylyshyn refers to as the "functional
architecture" of the mind. Pylyshyn most frequently refers to
the functional architecture as those fixed capacities or fixed
mechanisms that are "instantiated in the biological medium"
(Pylyshyn, 1980, p. 126) or "can only be explained biologically"
(Pylyshyn, 1980, p. 127). These fixed capacities can not be the
result of true learning, or deriving knowledge about the
environment (Pylyshyn, 1984, pp. 266-269). In terms of a
computational model, the operations performed by the functional
architecture correspond to the "wired-in functions" of a computer
(Pylyshyn, 1979a, p. 21). Hence, if any empirically-observed
psychological phenomenon is purported to be a root phenomenon due
to a fixed capacity, it must be one that is stable, invariant,
and not the result of applying tacit knowledge.
In light of the above considerations, Pylyshyn says that the
imagery debate should be understood as one concerned with
...whether certain aspects of cognition, generally (though
not exclusively) associated with imagery ought to be viewed
as governed by tacit knowledge -- that is, whether they
should be explained in terms of processes which operate upon
symbolic encodings of rules and other representations (such
as beliefs and goals) or whether they should be viewed as
properties of certain representational media or of certain
mechanisms that are not alterable in nomologically arbitrary
ways by tacit knowledge. (Pylyshyn in Block, 1981a, p. 153)
In short, the empirical search is for a non-computational,
unalterable unit in cognitive processing. Since images, qua
visible presentations, are non-computational, this search amounts
to one guided by the following supposition:
If an experiment can show the existence of any irreducibly
visual stimulus-like properties in mental imagery, this
suffices to show that imagery can be a root process in
cognitive psychology.
This way of stating what is as stake has proven satisfactory to
both parties. If any such process is indicated, Kosslyn's view
is supported. If no such process is found, Pylyshyn's view that
there can be root processes, but that imagery processes are not
presently found to be among them is supported.
The search for a true imagery process is sometimes
identified as the search for an "analog" process, and this is
also an acceptable description, if properly understood. From the
computational standpoint, this means the search for a process
that could not be accounted for by means of more primitive
computational processes. Every computer has wired-in features
that are impervious to alteration by programs; these features may
be called "analog" simply because of this. Alternately, "analog"
may be used to characterize the nature of a particular kind of
input/output function. Imagists have often used the term
"analog" to designate processes that exhibit the "smoothness" or
"continuity" in functionality that is thought (in their theory)
to correspond to operations due to imagistic processes rather
than due to discrete (descriptive) processes. Although
"smoothness" is not a necessary attribute of any given primitive
operation per se, there is nothing wrong with assuming that some
primitive operations are also "analog" in this extended sense as
well. Imagists, however, have to show that any reputed "analog"
operation is not only primitive, but also that it truly retains
some irreducible properties of a visual stimulus.
There are other ways of describing what is at stake in the
empirical side of the debate, but I take the above to be the most
philosophically important one.
II. What Kind of Phenomena are to be Explained?
Consider the question:
(1) Do German shepards have triangular-shaped ears?
Some people might answer straight off "yes," because they know
that having triangular-shaped ears is just a plain fact about
German shepards. But many people answer such a question only
after a moment of reflection. They report that this information
is something that is checked by means of inspecting a visual
mental image. Apparently, some people have information stored in
a form that allows immediate verbal responses, while others
derive the same information from the interior inspection of an
image. Their belief is apparently in the more ambiguous form of
an image, from which a verbal description may be derived.
Now consider the following examples.
(2) Which is larger, an ant or an iceberg?
(3) Which is larger, a mouse or a hampster?
(4) How many windows are on the front of your house?
(5) What letter of the alphabet does the capital letter "n"
look like when it is rotated 90 degrees counter-
clockwise?
(6) Imagine a cube. Now "paint" one side blue. Paint two
sides that are adjacent to the blue side, but opposite
each other, red. Now cut the cube into nine smaller
cubes (as in a Rubik cube). How many of the smaller
cubes have exactly one red side and one blue side?
(7) Imagine that your house has a doorway facing the street.
You are inside the house, ten feet from the open
doorway. The threshold of the doorway is level with the
street outside, which runs perpendicular to your line of
sight out through the door. At this moment, a huge
black car tire, ten miles in diameter, begins rolling
slowly down the street from left to right. As the tire
passes your house, what do you see through the doorway?
All of these examples are drawn from the contemporary literature
on mental images. On the average, people will find that as they
proceed through the questions imagery becomes progressively more
involved in solving the problem. Question 2 usually induces no
imagery. Question 5 nearly always induces an image, and most
people are successful at using the image to arrive at an answer.
In question 6, it seems that forming a mental image is absolutely
necessary. Many people, while recognizing this necessity, are
still unable to mentally maintain and manipulate an image that
will provide a solution. Finally, question 7 presents a task in
which imagery seems to be required, but the solution is elusive
even with an image.
Contemporary empirical cognitive psychology seeks to
generate structure/process models that will correspond to the
measurable behavioral responses people make when faced with such
cognitive tasks. It is not, except peripherally, concerned with
describing, investigating, or recording the subjects'
introspective accounts of the phenomenal states that occur during
such experiments. Subject reports are not considered actual data
under the rules of this form of experimentation; they are
considered supplementary confirming evidence.
III. Influential Studies
In this section, I review two of the most-often cited forms
of experiments in the literature. I show how the apparent case
for imagism can be undermined by descriptivist concerns.
A. Mental Rotation
1. Shepard and Metzler
Shepard and Metzler's 1971 experiment is the most famous
example of an attempt to isolate and quantify a specific
cognitive ability involving mental imagery. This experiment and
the hundreds of subsequent studies on mental rotation are
central elements in the empirical foundations of today's
cognitive psychology.
In the experiment, subjects were shown line drawings of two
simple three-dimensional shapes. Subjects were instructed to
find if the figures were congruent on non-congruent (the "same"
or "different"). In half the cases the shapes were identical
except for a rotation in space. The shapes were rotated with
respect to each other either in depth or in the picture plane.
In the remaining half, the shapes were mirror images and could
not be brought into coincidence through any possible rotation.
Example shapes are shown in Figure 5-1.
Eight male subjects were shown 1600 pairs of shapes. Each
subject was tested in several test sessions of about one hour,
for a total time of 8-10 hours. The reaction times for each
subject to perform the task (in the case of matching pairs only)
were averaged.
It was discovered that the average time it took subjects to
identify which shapes were identical was directly proportional to
their angular displacement from each other. The difference in
times for figures rotated in the picture plane instead of in
depth was negligible. The results are shown in Figure 5-2.
Initially, Shepard and Metzler were careful not to make expansive
claims and simply stated "the result places severe constraints on
possible explanations of how subjects go about determining
identity of shape of differently oriented objects" (Shepard and
Metzler, 1971, reprinted in Beakley and Ludlow, p. 216).
Subsequently, (1976) Shepard and his associates claimed that such
experiments demonstrate "the analog nature of mental rotation"
(Cooper, 1976, reprinted in Shepard, 1982, p. 170). The linear
function strongly suggested causal mechanisms that operated with
machine-like regularity and predictability, causing the mental
rotation to be accomplished by a repetitive process through which
the mental representation was "rotated" in minute increments (see
Cooper, 1976, reprinted in Shepard and Cooper, 1982, p. 160).
Kosslyn, Shepard and many others embraced this result as an
indication of the reality of irreducibly imagistic processes.
2. Learning and Mental Rotation
If there is significant improvement through practice
demonstrating acquired knowledge about how to perform a task such
as "mental rotation", then the task is cognitively penetrable
(see Pylyshyn, 1979a, especially pages 21 and 26). If this
improvement with practice, or what I shall in this instance refer
to as "learning," is involved to a significant extent in the
Shepard and Metzler task, then the experiment is not a true
indication of an isolated, holistic, analog process, and it does
not count as evidence against Pylyshyn's side of the debate.
Although I later criticize the over use of chronometric studies
(similar to Shepard and Metzler's) as theory-building tools, it
can be shown that even by the standards of evidence of these
experiments, there is not a good case for the existence of a
fixed, or "hard-wired," mental rotation mechanism that applies to
a wide variety of stimuli.
Yuille (1983) was among the first to introduce the issue of
learning and, by expanding the parameters of the original Shepard
and Metzler experiment, to show serious flaws in their
interpretation. An important historical footnote should be made
here. By the time Yuille made his criticisms, the cognitive
psychology movement, particularly the imagist side, had gained
such momentum that Yuille's work was ignored. Kosslyn, in fact,
did not even mention Yuille until 1994 when he (Kosslyn)
officially gave up chronometric studies.
First, Yuille reasoned that if rotation were a fixed process
operating on holistic entities at a fixed rate, the figures
should be rotated at the same rate even if rotated in memory
only, without the presence of a stimulus figure. Yuille trained
subjects to perform a memory version of the Shepard and Metzler
task. Subjects were shown a figure and its mirror image and were
given time to memorize the figures. Subjects were then presented
with a series of drawings and asked to identify if the drawings
were the same or different than the memorized figures. Yuille
found that in the memory version of the task the overall speed of
rotation was nearly five times faster than the visual version in
which figures were present. Yuille concluded that mental
rotation, if it was in fact a holistic process proceeding at a
uniform rate, had to allow at least two versions: one involving
present perceptions only, and one in which the rotation occurs in
memory.
Second, Yuille also performed experiments to show how
subjects' knowledge of the task altered the results. During his
memory experiments, Yuille had discovered that if subjects were
not given the mirror image of the figure to memorize, their
response times in the memory task were doubled. Evidently,
subjects were deriving information from the study of the mirror
image that significantly influenced their performance on the
task. Since knowledge of figures not seen altered the process,
this was not consistent with the idea that a holistic
representation of the figure, derived entirely from the current
stimulus object, was the element responsible for cognitive
processing. This led Yuille to doubt that "mental rotation" of a
single representation occurred at all.
To further test the hypothesis that ancillary information
could influence performance, Yuille informed subjects of a way of
perceptually dissecting the Shepard and Metzler figures so that
they could perform the task by inspecting only the bottom half of
the figures. The informed subjects performed the standard
Shepard and Metzler task approximately twice as fast.
Finally, Yuille conducted additional experiments with
complex figures and found that the complexity of the figures also
significantly influenced rotation times. He designed figures
with more cubes and more articulated segments. When the figures
became too complex, some subjects could not perform the task at
all. Those that could perform the task with complex figures
invariably found ways of simplifying the task by visually
examining only part of the figure. Yuille concluded that in this
case "imagery in a simple, holistic sense, does not mediate the
so-called mental rotation task" (Yuille, 1983, p. 278).
Yuille argued that Shepard and Metzler's results were
obtained because of a fortuitous selection of the figures used.
The shapes used were neither too simple nor too complex. Since
he did not find evidence of holistic rotation in his own
experiments, Yuille speculated that the Shepard and Metzler task
itself was not performed in a holistic fashion: "it is our
assumption that a piecemeal comparison process mediates this
task" (Yuille, 1983, p. 278). He suggested that the increased
time for increased angular disparity is accounted for by the
number and distance of eye movements required for feature
comparison between two figures (Yuille, 1983, p. 278).
Since Shepard and Metzler averaged the data of all subjects
over many hours, their finding also does not address the degree
to which early learning stages differ from the average. Their
supposition is that averaging results over many hours of testing
yields an accurate picture of the mechanism to be isolated.
In order to examine this supposition, I devised an
alternative experimental procedure using the Shepard and Metzler
figures. With a great deal of assistance, I conducted a series
of tests on small groups of 10 or more students (26 students
tested in all). We used only 30 sets of picture pairs instead of
800 sets. Each subject was tested only 3 times each for each set
of angular displacements between the picture pairs (instead of 80
times, as in Shepard and Metzler's experiment). The procedure
was intended to capture more information about the initial phases
of the subjects' mental rotation.
We discovered that there is no indication of an fixed
rotation speed in the early stages of performing the Shepard and
Metzler task. As shown in Figure 5-3, subjects generally improve
their times after their first attempt to rotate an image at any
given degree of angular separation. For this particular group of
subjects, however, the first attempts at 140 and 160 degrees were
not as fast as subsequent attempts. This may have been due to
characteristics of the figures or to the order in which they
appeared on the test. We also kept records of how a single
individual performed on a slightly longer series of trials.
Figure 5-4 shows how a single individual dramatically improved
his times after two successive trials of 140 Shepard and Metzler
figures each. The first trial indicates the subject took
considerably longer with rotations greater than 120 degrees. As
indicated in the graph, in the second trial this deficiency is
overcome and a nearly linear average response time similar to
Shepard and Metzler's published results.
These data indicate that subjects can, with practice,
substantially improve their times in performing the Shepard and
Metzler task, particularly with the more difficult (more
angularly divergent) figure pairs. These data alone are not
sufficient to demonstrate "learning" in any more than the
limited, behavioristic, sense that the subject performed a given
task faster. Such data alone (as I argue in Section III of this
chapter) can be misleading and do not capture the kind of
evidence necessary for understanding the conscious component of
mental imagery problems and related phenomena. Fortunately, from
the point of view I wish to advance, I also collected subject
reports of how they approached the Shepard and Metzler task.
Subjects reported both conscious cognitive strategies of
decomposing and analyzing the figures as well as just
"superimposing" (presumably by means of a phenomenal image) the
figures. The combined data, including both statistical and
subject report data, do, in my view, constitute substantial
evidence that subjects adopt learning strategies during the task.
The importance of subject reports is a point we shall return to
later in this chapter. At present, it will be instructive to
consider still another so-called "mental mechanism" phenomenon
that was thought to be identified and measured by cognitive
science: mental scanning.
B. Mental Scanning
1. Kosslyn
Kosslyn's most famous experiment in mental scanning involves
a map memorization task. Subjects were asked to memorize a map
of an island with seven different objects on it (Figure 5-4).
None of the distances between any two objects on the map are
identical. Subjects studied the map until they could reproduce
the locations of all seven items to a specified degree of
accuracy. Accuracy of recall was tested by having the subjects
locate the positions of the objects on a blank piece of paper.
The practiced subjects were then asked to visualize one of
the objects on the map. The subjects were then given the name of
another (target) object. This target object was either the name
of an object on the map or the name of some other common object,
but not one on the map. Subjects were to mentally scan the map
in search of the target object. The inclusion of target objects
not on the map was to ensure that subjects would actually scan
their mental image of the map and not circumvent the instructions
by simply memorizing the names of all the objects on the map.
Kosslyn describes the instructions to the subjects and the
process of making measurements as follows.
Upon hearing this name, he or she was to "look for" the item
on the imaged map. If the item was on the map, he or she
was to scan to it; scanning was to be accomplished by
imaging a small black speck moving in a straight line as
fast as possible to the object. ...As soon as the dot hit
the second object, the subject was to press one button; if
the subject "looked" and was unable to "see" the second
object, he or she was to press another button. The subjects
were asked to respond as quickly as possible while still
following the instructions and keeping errors to a minimum.
(Kosslyn, 1985, p. 322, summarizing his original 1978
experiment)
Kosslyn found that the average times to mentally scan the
distance between objects on the map was directly proportional to
the distance between objects. The distance/time relationship
results in a straight-line function, shown in Figure 5-5. Kosslyn
concluded that the experiment verified his parallelism
hypothesis:
This demonstration supports the claim that images are quasi-
pictorial entities that can be processed and are not merely
epiphenomenal. One of the characteristic properties of such
a representation is that the interval distances between
portions are depicted, and these data suggest that
functional representations underlying the experience of
visual mental images do have this property. (Kosslyn, 1980,
p. 44)
Kosslyn's experiment was accepted by a large segment of the
cognitive science community at the time. Combined with the
Shepard and Metzler experiment, the straight-line functions of
mental operations came to be understood as evidence of analog
processes.
2. Pylyshyn's Objections to Scanning Experiments
Pylyshyn argued that mental scanning can be shown to exhibit
cognitive penetrability. If subjects are not specifically
instructed to mentally scan to the target object, they respond to
questions about whether or not other objects are on the map after
a fixed delay that is completely independent of distance.
Kosslyn himself reported this phenomenon (Kosslyn, 1980, p. 47).
Other experimenters (for example, Lea, 1975, as reported in
Kosslyn, 1980, p. 40) also found similar phenomena when subjects
were not instructed to use imagery.
Pylyshyn also pointed out that cognitive penetrability is
evident in the way Kosslyn describes his original experiment.
The subjects are instructed to scan toward the target object.
How can subjects carry out the instructions unless they already
know in which direction to "scan" toward the target object. The
direction in which to scan is knowledge that must be retrieved
nearly instantaneously, before the "scanning" process could
start. Therefore the subject already anticipates something about
the terminus of his "search." Pylyshyn proved this himself with
his own experiments in which he asked subjects to memorize a map
and then name compass directions between various objects. No
time/distance relations were evident in their responses (Pylyshyn
in Block 1981a, pp. 193-199).
Pylyshyn concluded that mental scanning is not a process in
which subjects actually find the objects in an image. Rather,
subjects engage in a process that is conceptually distinct from
any reputed searching for an object. What subjects do is not
anything that represents a constraint by the medium of
representation, but represents what subjects believe or infer
"about some likely intermediate stages of an event being imagined
or about the relative times at which the event would occur"
(Pylyshyn in Block, 1981a, p. 186). The supposition that a
medium is involved in or imposes constraints on the responses of
subjects is simply an ad hoc hypothesis invented to explain the
data.
3. Finke
If the scanning results and the Shepard and Metzler results
were questionable as solid evidence for the imagist side, then
the imagists needed empirical evidence that was more difficult to
refute. Finke attempted to supply this evidence in his 1982
experiment. He attempted to block Pylyshyn's form of objection
by devising an experiment that could not be shown to be subject
to task demands or to be cognitively penetrable. Currently,
Finke's experiment is often cited as proof that scanning
phenomena truly are analog processes. It is claimed that because
it is difficult to show how any task demands could influence the
experiment that it is exceptionally strong evidence that the
time/distance relation maintained in imagery task performances is
the result of the intrinsic constraints of the "mental medium."
These constrains would show that the medium maintains an
"analog," or smooth and continuous representation during certain
imagery tasks, a representation not subject to the intrusion of
discrete, or non-analog, computations. If Finke's experiment and
the claims he makes about it are indeed right, then one of the
principal claims of computational imagism gains significant
empirical support. (See Kosslyn, 1994, pp. 9-10 and Tye, 1991,
p.67 for expressions of this viewpoint).
Finke used a pattern of four circular dots black dots on a
white background as a stimulus display. Subjects were told to
memorize this pattern and were allowed to study the display for
five seconds. The four dots were then removed and the display
remained white for one second. This interval was to allow any
induced after-images to fade before the memory test was started.
A small arrow then appeared on the display, which either pointed
directly at the position of one of the dots or completely missed
pointing to any dot by an angle of 40 degrees or more. The arrow
appeared at an unexpected location, at various distances from the
dots. Subjects were instructed to respond as quickly and as
accurately as they could with a button press to indicate whether
the arrow pointed to a dot location or not. Subjects were told
in advance that the experiment involved visual memory and that
the task would be easy since the arrow would point directly at
the dots or miss it entirely. No mention was made of using
mental imagery.
Finke found that the average times to make the decision in
those cases where the arrow pointed directly to a dot, was
linearly proportional to the distance from the arrow to the dot.
Finke concluded that although other interpretations were
possible, this fact, coupled with some subjects' reports that
they formed mental images, made it most likely that "this kind of
task is typically performed by scanning along a straight line in
a mental image" (Finke, 1982, p.144).
Finke claimed the experiment measured the scanning
phenomenon in a way that made the use of prior knowledge
impossible. Although subjects might have encoded the various
distances between the dots, this information could not be used to
solve the problem since it involved the appearance of a new
element, the arrow, at an unexpected location. The problem
required establishing the relation of the arrow to the previous
visual information. The relation of concern to the subject is
"pointing to" or "not pointing to." This does not involve the
notion of distance. Since the subjects were not told that the
experiment involved mental imagery, scanning, decision time, or
distances it is unlikely that they would consciously or
unconsciously try to establish a correspondence between the
distance from the arrow to a dot and the time needed to traverse
the distance (i.e., form a conceptual relation corresponding to T
= D/S). For these reasons, Finke claimed that the issue of task
demands or tacit knowledge could be eliminated from this
experiment.
The results appear to be consistent with the pictorialist
view that images are not epiphenomenal but are functional in
problem solving. As Finke says,
In general, these interpretations are consistent with the
view that attributes to mental imagery the special function
of facilitating incidental recall and inferential reasoning,
that is, the extraction of information not explicitly
encoded at the time of initial learning. (Finke, 1982, p.
146)
The results also appear to be consistent with Kosslyn's theory
that the spatial characteristics of image processing are inherent
in the medium rather than derivative of beliefs or goals.
Finke's experiment is indeed difficult to discount, but
there is a descriptivist response to it. First of all there are
some omitted details to the experiment that have not been
consistently reported. In preliminary trials, Finke discovered
that if an arrow appeared at a point previously occupied by a
dot, the response times were independent of distances. This led
him to conclude that the relative interpoint positions were
remembered and there was no need to utilize a mental image in
this task. Finke also discovered that the reaction time/distance
ratio decreased slightly as subjects became practiced in the
experiment. Furthermore, only six of the twelve subjects
actually reported scanning a mental image along the axis of the
arrow, while five reported using an image "in some way," and one
reported not using images at all. This is why Finke hedged his
conclusion by saying the task "typically" involves imagery (see
quotation above). Kosslyn, in his references to this experiment
as a final blow to the issue of task demands (see 1985 and 1994
texts) does not refer to any of these important details about the
experiment. Finke does not report these details in his Scientific
American article, nor does Kosslyn in his 1994 book.
Far more important than these omitted details is Pylyshyn's
response. Pylyshyn responded that the entire experiment was
essentially irrelevant to imagery since it involves use of
specific visual features in a present stimulus (Pylyshyn, 1984,
p. 247). He writes:
When you are actually looking at a screen you can use
location cues such as the borders of the screen or specks on
the surface to indicate where to move attention (or even
your eyes) -- in which case this becomes a visual task and
not an imagery one. (Zenon Pylyshyn, personal
communication).
This analysis supports the definition of imagery phenomena given
in Chapter 3 and how to understand its experimental implications
(the fourth objective of the study of imagery types). Imagery
experiments need to be arranged so that stimulus conditions are
not confused with imagery phenomena proper (which, by the
definition we made in Chapter 3, involve effects attained in the
absence of a corresponding stimulus).
C. Conclusion
From our discussion so far, we conclude that although the
operations of the brain indeed must have some constraints, the
scanning operation and the rotation phenomena do not appear to be
among them. The spatial relations that appear to hold in these
cases are in fact optional in the crucial, empirically verifiable
sense that they can vary or disappear under alternate
experimental conditions. Hence, the scanning and rotation
phenomena do not appear to be sufficiently constrained to be
incorporated into cognitive psychology as root phenomena.
IV. Problems for Empirical Cognitive Psychology
A. Individual Variation
As long as averages are adduced as evidence of what
individuals are actually doing in a given task, the experimenters
run the risk of making interpretive hypotheses that apply only to
ideal responses. Instead of applying to everyone, a hypothesis
derived from an ideal response may apply to no one. As Darryl
Bruce pointed out in an essay critical of contemporary laboratory
approaches to memory, the over-use of averages implies a type of
Platonism about psychological entities. This approach is an
inversion of the perspective he recommends. Rather than
supposing that mean values are real, Bruce suggests "it is
variation in the group that is real and the mean value or type
that is the abstraction" (Bruce, 1985, p. 82). The danger is
that the features of an idealized average will become the object
of psychological investigations. This may obscure what is
happening on an individual or finer-grained level of analysis.
We saw how the Shepard and Metzler results did not necessarily
indicate all that subjects were doing, and how the interpretation
of the experiment changed once it was subjected to a finer-
grained analysis.
The problem of how to incorporate individual variations
within psychology, particularly when one is searching for root
phenomena, is absolutely fundamental for experimental psychology.
Hull, one of the most important theorists of behaviorism,
explained variations in response to an identical stimulus by
supposing that they are caused by random oscillations of neurons
about a central level of activity. He concluded that although
this precluded an exact predictive science of individual
behavior, repeated measurements would be sufficient to "reveal a
close approximation of the laws which are operating" (Hull, 1943,
p. 317).
A less restrictive way to account for individual variations
in task performance is to allow that subjects employ different
strategies for the same task. My experiments with mental
rotation indicated this from an objective standpoint, but the
easiest way to clear the matter up is simply to allow subject
reports into the "data" of the experiment. If we assume that
subjective reports are fairly accurate, we get a whole new and
easily-explained set of data consistent with the view that
"percepts" do not come in pre-formed computational (or non-
computational) units, but in ways varying according to subject
dispositions and interests. It may be that subjects who report
piecemeal solutions actually employ them, while others who report
holistic solutions may actually employ them. Perhaps what counts
as a unit of processing, either on the conscious or the
unconscious level, for one person does not count as a unit of
processing for another person. This would account for individual
differences in times, and, since the various strategies will be
distributed over different subjects as well as the same subjects
in different iterations of the task, the overall result will be
an "average" performance corresponding to some mathematical
function (in some cases, conveniently, a linear relationship).
B. Underdetermination and Lack of Predictive Power
Kosslyn and Pylyshyn have accused each other of having an
unscientific theory that lacks predictive power. Let us briefly
review the bases for such accusations, and then suggest a
possible resolution. (In order to frame the accusations, we need
to assume, for the moment, that the above considerations about
individual variation are moot, and that an interpretation of
Platonized data is needed.)
Kosslyn argues that Pylyshyn's theory can not explain or
predict the rotation data. If Pylyshyn's theory is right, there
are no constraints due to an internal spatial medium. But if
that is the case, why has one "rotation" rate been measured
rather than another? Pylyshyn's response is that there is a
rotation parameter in cognition that is set by the "cognitive
analysis of the stimulus and the subjects beliefs and goals"
(Pylyshyn in Block, 1981a, p. 178). Kosslyn counters by saying
that the explanation that the parameters of the experiment
somehow "set" the rotation rate is simply an ad hoc hypothesis.
There is no reason to suppose that this is a knowledge-driven
response rather than one reflecting the properties of the
internal representational medium. In his 1994 book, Kosslyn took
the stand that the entire descriptivist side had the character of
being an ad hoc system; he called it "a moving target" (Kosslyn,
1994, p. 8). Whatever data was found, he pointed out, could be
adapted to the propositionalist side simply be proposing new
"knowledge bases" or "parameters" sufficient to cover the data.
On the other hand, Pylyshyn has pointed out the kinds of
problems faced by Kosslyn's stance on his map scanning
experiments. Kosslyn admitted that subjects could find or "see"
the target objects independently of distance, but he insisted
that this was evidence for his position. When subjects choose to
follow the instructions, the scanning time/distance relation
emerges. When they do not, the phenomenon does not appear.
Kosslyn concluded that this is evidence that imagery, when used,
has certain spatial characteristics. One can only regard such a
defense as a last ditch effort to maintain credibility where
there is none. A task demonstrating a root imagery phenomenon
can not optionally involve imagery processing; this violates the
principle that an experiment must involve irreducibly visible
stimulus-like properties in order to be proof of imagery
processing, as set forth in the introduction to this chapter. As
Pylyshyn has argued, in order to be sure that any phenomenal
imagery state associated with a task is not just an epiphenomenal
result of a descriptive process, it must at least be shown that
the task can not be done in any other way than through generating
a phenomenal image. Once this has been established, one can then
debate whether or not the characteristics of the task demand that
new information has been extracted by means of the phenomenal
image.
One reason, I suggest, that neither side has been able to
get the upper hand on this issue is that computational models are
simply too flexible. So flexible that, given the original terms
of the debate (i.e., assuming some form of computationalism to be
true) it may be unsolvable in principle. This idea was taken up
by Anderson in 1978. Both sides, he pointed out, agree the
problem is to identify the form of the mental representations and
the processes that operate on them. The representations and
processes always work together, forming the complete system that
is to be investigated. Anderson argued that given any
behaviorally-measured data, it is possible to devise a model
operating either with images or propositions as the root form of
representations. Either form of representation would serve
equally well, since appropriate processes can be proposed to
accommodate them. If, for example, the root representations are
considered as holistic images, then a process, say, of rotation,
can be hypothesized to operate on them. If, on the other hand,
the representations are thought to be descriptions of positions
and angular quantities, then a process of calculation can be
hypothesized to operate on them.
Since the behavioral data can always be interpreted as
conforming to some combination of representations and processes,
it will never be possible to isolate the specific nature of one-
half of the operating pair (either the processes alone or the
representations alone). Anderson suggests, therefore, that the
debate is essentially insoluble because the two theories are
computationally equivalent. Cognitive psychology will be left in
the position of proposing various models that emulate rather than
capture the nature of mental phenomena. He compares the
situation to that in physics, in which the same phenomenon may be
described in terms of either waves or particles.
Anderson's critique, I believe, is basically correct given
the assumptions it makes. He accepts Kosslyn's idea that images
could be computational entities, but he correctly diagnoses the
problem of working with hypothetical models that are constrained
only by the process input, the process output, and a time lag in
between. This is just a black-box problem. Given only the input
and output, any series of events can be hypothesized to mediate
them, limited only by the creativity of the model maker.
The supposition that there are any serious constraints on
the mediating model derives, I suggest, from a predilection to
accept as a supposition precisely what is to be proved. The idea
that computational images or descriptions, in whatever preferred
combination, actually mediate input and output derives not from
computational restraints, but from the actual phenomenology of
conscious imagery experiences. These, as we know, involve both
active and passive, willed and responsive, consciously-directed
activities and some measure of automatic awareness of spatial
environments. The model maker just attempts to solidify these
global aspects of conscious mental life that actually mediate
perceptual problem-solving and imagery tasks into a computational
model. Since the model maker can not capture the nuances of
conscious problem-solving, the model fails.
Some of the misunderstandings about the value of cognitive
model-making derive from the tension between two issues that
underlie Kosslyn's approach: on the one hand he seeks to show
that the phenomenal image is involved in causing the subject's
response, on the other he seeks to show that there is a special
medium that constrains the time of the response. Is there a
perspective from which we can gain information about the nature
of imagery experiences, including whether or not phenomenal
images were present and used by the subject? Of course. One has
only to perform mental rotations or the Finke experiment on one's
self and then ask others to do it and report their findings. To
be more rigorous, one might use more demanding tests, such as
those designed by Stromeyer and Psotka or Chambers and Reisburg
(section V of this chapter) from which the presence of a
phenomenal image seems even less logically intractable. But any
claims or implications of this sort made by Kosslyn are
independent of the (in his view) more important claims he makes
about what interpretation the hard evidence alone demands. On
this issue, I side with Pylyshyn. The present evidence is simply
insufficient to make the claims. Further, Pylyshyn's claims to
the contrary notwithstanding, I think Anderson's critique of the
standards of evidence and the models used to explain the evidence
stands. Suppose we had a model of human imagery processing that
was fully accurate and predictive of the behavior of subjects in
experimental conditions. Are we then constrained to attribute to
the subjects the same functional states as are used in the model?
We would be tempted to, naturally. We would not, I would argue,
be constrained to, unless there were other evidence, such as
neurophysiological evidence or subject reports conforming to the
model. Kosslyn's subsequent claims that we do have such
confirming evidence (an I do not dispute its value, since it is
part of the descriptive model of traditional/common-sense I
support) should not be confused with specific claims he makes
about interpretations forced on us by specific experiments.
Restricting ourselves to the input/output/time-lag model
that Anderson criticizes, could there be any sort of task
measurable in an experimental situation that will demand that
"imagery processing," rather than "descriptive processing," be
included as the hypothesized mediating component in cognition?
Can we exclude this as a possibility at this stage of the game in
empirical investigations? Perhaps not. Perhaps in the future
some sort of test will be formulated that can demonstrate one
sort of interpretation of the evidence in favor of the other.
Both Pylyshyn and Kosslyn strongly disagree with Anderson that
the issue is undecidable in principle. Indeed, if Pylyshyn is
right, and the experimental arm of cognitive science succeeds,
there will be a discovery of a root mechanism in the functional
architecture (loosely termed an "analog" or perhaps even an
"imagistic" process). Pylyshyn's point about the present state
of the empirical data, and the one re-emphasized here, is that we
have very little in the way of unequivocal data (about mental
imagery, as opposed to vision, or other areas investigated in
cognitive science). There must be, according to Pylyshyn,
cognitively impenetrable elements in the functional architecture,
but so far, there is no evidence that the scanning and rotation
phenomena, as empirically defined and measured, are among them.
C. The Will and Conscious Judgment
If the aim of psychology is to reduce the phenomenal image
to some other explanatory mechanism, the phenomenal image has no
role in science. This view, which we have seen is supported by
both Pylyshyn's and Kosslyn's theory, has a strange consequence
for the Shepard and Metzler experiment. What if I experienced no
images, no thoughts, and no subjective experiences during the
experiment? Under these circumstances, would I be able to arrive
at anything that could rightfully be called a judgment?
Let us consider this for a moment, for many of our
intuitions about consciousness, thought, and the prospects for
computational solutions in the philosophy of mind depend on them.
Idiot savants have been reported to be able to multiply large
figures in their heads. The answer may be correct, but is it
right to say that the subject thought of the answer? Did the
subject think it through? According to the reports of such
subjects, the answer is "no." The answer just "appears" in
consciousness. According, then, to the traditional philosophic
definition of knowledge an idiot savant has no knowledge of what
he is doing, since knowledge is true belief plus the ability to
give a reasoned account. When we are considering models of human
processes, we need to bear our intuitions about such examples in
mind.
Let us return to the image rotation example. If I reported
that my first awareness was the stimulus (the figures placed
before me) and the next awareness I had was my finger pressing
the "SAME" button, would anyone be likely to interpret the
experiment as showing how we perform mental processes? I think
not. Yet, within the prescriptions of contemporary techniques,
the total absence of these intervening mental states is
completely compatible with the purpose of collecting data from
subjects in order to form straight-line graphs indicating the
nature of the mental processes they use. This is a strange
consequence because it implies that the ordinary meaning of
"using a mental process" can completely disappear in a scientific
context.
Consider the psychological phenomenon of the perception of
color constancy. Two identical sheets of paper, one in bright
light and the other in shade, can be easily and effortlessly
judged to be the same color. The neurological processes and the
principles of light reflection involved in this phenomenon are
undoubtedly complex, yet our subjective experience involves
virtually no effort. Problems in rotation, on the other hand,
require both effort and consciously-directed attention. I
suggest that an unprejudiced, common-sense view of the matter is
that the forms of explanation required in the two cases diverge.
Once the physical explanation of color constancy is given, we
need an additional explanation of the presentational
consciousness of color; that is, we need to explain how something
can appear to us in a certain way without conscious effort.
Rotation problems require an additional explanation of how
problems requiring effort and self-direction can be solved. The
continuity of an interconnected series of presentations in
consciousness resulting in a judgement needs to by explained.
Behaviorist-inspired psychology attempts to collapse the
difference in modes of explanation by collapsing the difference
in the subjective experience in those processes requiring effort
and those processes that occur automatically.
D. Conclusion
The various attempts we examined to find a root imagery
phenomenon did not, upon examination, yield a consistent picture
even within the strictures of computational models. We gave
ample reasons to question the fruitfulness of such models.
We are now ready, hopefully, to entertain another
possibility about the "kinds" of information encoded in the brain
and available to us. Perhaps the search to verify the existence
of "pure" analog or discrete coding information is not a problem
of experimental design. It may be that the lack of separation of
the two supposedly distinct "kinds" is a feature of the
information itself, and that this is the reason experimental
evidence tends to be equivocal. This might be revealed in other,
less restrictive, experiments.
V. An Alternate Experimental Model: Chambers and Reisberg
Chambers's and Reisberg's approach is distinctive in that it
integrates elements from both Pylyshyn's and Kosslyn's theories.
In support of a descriptivist perspective, Chambers and Reisberg
claim to show that unprompted "discovery" of new interpretations
from a single image does not occur. Yet, in support of a
pictorialist perspective, they also claim that "learning" or
surprise may be present in mental imaging. The latter claim is
to be understood advisedly, since they show "learning" takes a
form that is consistent with these conceptual constraints built
into the image and therefore does not represent learning from a
neutral stimulus.
A. Experiments
Between 1985 and 1991 Chambers and Reisberg reported on a
total of eight experiments using various ambiguous figures that
could be reinterpreted by reversing front and back, rotating the
figure, or reversing figure and ground. A total of some 200
subjects were tested in these experiments. In many cases,
subjects were given practice sessions in which they were taught
to reverse standard ambiguous figures such as the Necker cube or
the face/vase figure. In a typical experiment, subjects were
allowed to inspect a stimulus, such as the duck/rabbit figure,
for five seconds. This was generally long enough for a subject
to arrive at one interpretation of an ambiguous figure but not
the other. Subjects were then given instructions (in the manner
of many of Kosslyn's experiments) to inspect and/or rotate their
mental images to find a new interpretation.
The results of these experiments were dramatically uniform:
in all cases except two, no subject was able to arrive at a new
interpretation of their mental image through untutored
inspection. (See Chambers and Reisberg, 1991, p. 345 for this
summary result. Chambers and Reisberg attribute the two
anomalous cases to lucky guesses on the part of the subjects.
These anomalous cases amount to only one percent of their data.)
In particular, none of the 35 subjects exposed to the duck/rabbit
stimulus in the original 1985 experiments were able to reconstrue
their mental images as representing another animal. Even when
subjects were specifically encouraged to find an alternate
interpretation of their image as they had previously practiced
with stimuli, they were unable to do so.
B. Qualifications of Results
These results need considerable qualification. Although no
spontaneous reconstruals were recorded, Chambers and Reisberg did
find that if subjects were given the appropriate hints or
instructions, they were able to give alternate interpretations to
the stimulus figure using memory information only. The way in
which these alternate interpretations became "available" (for
want of a better term) to subjects is of central importance if
Chambers and Reisberg are to maintain their claim that no
reconstual of images is possible. Despite the fact that their
results are strong arguments that mental images do not retain all
the properties of a neutral stimulus, the fact that new
interpretations are possible under some conditions argues that
some stimulus-like properties are retained.
An overview of the types of instructions used and how this
changed the results of Chambers's and Reisberg's experiments will
indicate the kind of dilemma they faced in trying to state a
consistent and comprehensive conclusion. Various forms of
suggestive instructions were tried, with no readily predictable
effects. The instruction to rotate the image, for example, did
not allow subjects to find an alternate interpretation, but the
instruction to give the figure "a new top" (i.e., think of the
side of the image as the top of the figure) often allowed
subjects to find an alternate interpretation. Similar results
were found using the duck/rabbit figure. Suggestions inspired by
the idea that mental images can be scanned, such as "look at the
left side of your image," or "this figure resembles a familiar
form" did not allow subjects to find an alternate interpretation.
But in latter experiments, Chambers and Reisberg (1992) reported
that rather specific instructions, such as "some people interpret
this figure as a duck" or "imagine the front of the figure as the
back of another" did allow a significant percentage of subjects
(about half) to find an alternate interpretation of the
duck/rabbit figure. Perhaps even more surprising, Chambers and
Reisberg found that the new interpretation apparently restored
visual information that had been lacking. If subjects originally
encoded the figure as a duck, for example, they were able, after
the new interpretation, to recognize visual details on the
"rabbit" side of the original stimulus figure with greater
accuracy.
Kosslyn also addressed the Chambers and Reisberg results and
their interpretation. In defense of a stronger imagist view, he
opposed their implication that images were fixed by descriptive
constraints that limited or rendered impossible spontaneous
reconstrual. Kosslyn pointed out that other data exists on the
abilities of ordinary subjects to reconstrue images. The
following exercise (slightly modified here) was used by Finke,
et. al. (1989, as reported in Kosslyn, 1994, pp. 336-337) to
demonstrate interpretive reconstrual:
1. Imagine an uppercase letter "d."
2. Rotate the letter 90 degrees counter-clockwise.
3. Place the letter of the alphabet following "i" on the
bottom of the image.
4. What does the figure resemble?
Most subjects have no difficulty describing the figure thus
constructed in the imagination; "an umbrella" is a typical
response. In this case, a figure was rotated and shapes are
assembled for the purpose of constructing an image. It seems
clear (phenomenologically, at any rate) that the resulting image
is then reconstrued as representing something other than the sum
of its parts.
Chambers and Reisberg attempted to bring the data together
and address such apparent counter examples by distinguishing
between learning and discovery within single images. The
Finke/Kosslyn "umbrella" example is a case of learning because
the subjects make a single image from separate images and learn
how that new image could be described. Within that new single
image, however, there is no authentic discovery, in the sense
implied by visual inspection of raw data. The new image also
brings with it certain unconscious parameters that restrict how
the subject will reconstrue the image. As evidence of this,
Chambers and Reisberg point out that there is no uniform
explanation or pattern that predicts how, when, or if subjects
will be able to reconstrue images -- as there would be if
Kosslyn's pure stimulus view were correct. It might be, for
example, that the umbrella case allowed "learning" because
everything in the figure is upright. This hypothesis fails to
carry over as an explanatory pattern to the duck/rabbit figure,
where everything is also upright and no spontaneous reconstual
occurs. In addition, some figures do not appear to exhibit any
preferential orientation that determines initial encodings: the
shapes of automobiles or scissors, for example, are instantly
recognized regardless of orientation. Chambers and Reisberg
concluded that more research might reveal a pattern, but
Kosslyn's pure stimulus view seems to be blocked.
C. Conclusion
In Chambers and Reisberg we have finally encountered an
approach that attempts to accept conceptual, experimental and
phenomenological evidence on more or less equal footings.
The Chambers and Reisberg experiments do not fit neatly into
either the pictorialist or the descriptivist camps of
experimental psychology. Images are not like neutral stimuli, we
are told, but then in certain circumstances, they seem very much
like them. There should be a theory explaining why some image
reconstruals are possible and others are not, but we presently
lack such a theory. Chambers and Reisberg strongly disagree with
Kosslyn's claim that images are pictorial in the pure stimulus
sense:
Images are not quasi-pictorial in the sense of needing some
interpretation. Instead, an image's referent is specified
by the imaginer. Thus although an image might in some
fashion resemble more than one content...this does not
result in ambiguity because it is not by resemblance that
the images refer. (Chambers and Reisberg, 1985, p. 325; also
in Clark, 1988, p. 56)
On the other hand, Chambers and Reisberg, appeal to the
pictorialist notion that images, i.e., phenomenal images, depict.
The phenomenal image has a certain (phenomenal) size,
orientation, color, and may incorporate elements that allow us to
learn from it. How do we derive information from the phenomenal
image such that we may be said to learn form it? Here is
Chambers's and Reisberg's (1992) proposal:
We propose that, in imagery as in perception, the specific
meaning of an image guides how attention is deployed across
the image. Thus, which aspects of an image will be clear
and which will be vague will also depend on the specific
meaning of the image. In this way, the meaning of an image
does not merely accompany the depicted geometry; instead the
meaning literally shapes the depiction. (Chambers and
Reisberg, 1992, p. 151)
Note that this way of stating the role of the phenomenal image
incorporates two conflicting views that have shaped the
contemporary imagery debate. One aspect of this explanation
supports a pictorialist interpretation. The subject directs
inward attention in order to "see" parts of the image that are
"clear." This is, of course, the vision metaphor of imagery that
Pylyshyn is so much opposed to. The mind's eye is presumed to be
able to pick up clear or vague information depending on the
quality of the image. It is implied that to some extent the
image in this role is passive -- for it is the attention of the
agent that accounts for picking up information from the image.
The explanation also incorporates nuances that would support
descriptivist theory. The meaning of the image is said to
"shape" the depiction. How are we to understand the term
"meaning?" Could the meaning be reduced to a propositional or
amodal form? Although Chambers and Reisberg attempt to use terms
such as "mean" or "understand" in a restricted technical sense
(applying to geometric specifications or other empirically
discoverable restraints to these terms) no pattern of the
meanings (in the technical sense) appears to be at hand.
Moreover, Chambers and Reisberg also state that the meaning of an
image, in the sense of intentional meaning, is something
designated by the subject and is irrelevant to the depictive
appearances.
In other words, computational considerations and the
abstract functional image, reputed to be the underlying
explanatory mechanisms, fade from the picture. In the final
analysis, we are left with an intentional psycho-physical view,
where the phenomenal image is deemed to be an amalgam of
conceptual and quasi-perceptual stimulus properties that
differentiate it from vision, and where the phenomenal image is
deemed to be the mediating cause in the entire imagery process.
Just as in traditional psychology, there is still empirical work
to be done, because not everything is given immediately and
infallibly through introspection. What is not revealed by
introspection, and has to be experimentally confirmed, are the
specific ways in which our mental images are preinterpreted.
These may well derive from neurophysiological limits of the ways
in which we can represent non-present spatial environments to
ourselves.
VI. Concluding Remarks
These remarks summarize some of the points made in this and
the preceding chapter. Here, I bring together some of the common
themes from these chapters, and incorporate them into some
reflections on the limitations of computational schemes in
general. Although only two computational theories were
investigated in detail, I believe that some of the results
obtained in my investigation are much more general, and are
likely to apply to other schemes, particularly those that (1)
employ the same sorts of understanding of what constitutes a
computational process and (2) use similar standards for what
constitutes data in an experiment. This is not to say that the
demands of scientific rigor ought to be abandoned, but only that
(as we saw in the Chambers and Reisberg approach) some
accommodation to philosophic ontology and phenomenology of mental
images be made. In addition to these concerns, I raise, once
more the background issue (from the beginning of chapter 4) of
studying imagery by means of theory that abstracts imagery from
consciousness and carries with it a strong commitment to
epiphenomenalism.
A. Theory and Experiment
Our investigation (in chapter 4) indicated that the most
consistent position for a computational theory is to hold that
images, in any ordinary sense of the word, can not be
incorporated as computational elements into the system. In as
much as the causal flow of information in a computer must be
attributed to the properties of the computational elements in it,
there will be no causally-effective imagery processes. As a
result, any imagery experiences that are thought to be caused by
such processes will be epiphenomenal.
The experimental approach to finding imagery processes
within a computational framework (examined in chapter 5) fared
little better. The experimental evidence we examined is
inconclusive at best, and possibly inconclusive in principle,
given the structure/process pair model and the limitation that
subject reports are not actual data. Releasing the latter
stricture yields a better result, and is consistent with many
aspects of traditional philosophic psychology.
Overall, however, both the theory and the experimental
approach are aimed at a very limited class of the kinds of
imagery phenomena discussed in Chapter 3. Most of the
experiments (not just the ones we reviewed, but those in the
entire literature) pursued in this vein investigate phenomena
that mix perceptual problem solving and short-term memory
phenomena. They do very little to establish what images are or
how they are involved in spontaneous imagination or thought. The
experiments are response-driven, and tend to turn images into
"objects" in the very senses we sought to avoid in Chapter 3.
The image becomes an object (no quote marks needed) of study
through behavior, an object of inspection through computational
processes, an unconscious physical brain representation, and so
on, rather than an "object" (quote marks needed) created in and
through an intentional act of the subject. As far as I know, no
contemporary, computational theory or experiment has given an
account of how one can spontaneously create images of a trip to
the moon and back.
The two contemporary theories we examined appear to have
little predictive value. For example, the rates at which mental
rotation or mental scanning occurs have not been found to be
highly predicable. If more research were done on individual
variation and subject reports were allowed as data, the results
would, I suggest, probably be better. For example, percept
variability (in terms of the amount and kind of information
picked up in each eye movement) and specific background knowledge
(e.g., knowledge of isomers in chemistry) could be correlated
with performance on image rotation tests.
B. Epiphenomenalism
Epiphenomenalism is like Cartesianism: it is a view people
often cling to with a kind of religious fervor, and, like
Cartesianism, it is probably impossible to disprove. One can
only point to the consequences of it and hope that believers will
be moved to question their faith. I mentioned at the beginning
of Chapter 4, that epiphenomenalism means there is no causality
in the mental to physical direction, and that as a consequence it
is technically incorrect to say that I move my finger -- what
really happens is that my finger is moved, not by me, but by
purely physical causes that have nothing to do with the mental
causal powers I normally attribute to myself. The same follows
for images. The mental images sports stars use to train
themselves would not be causally effective, my imagining that I
see red would not be the actual cause of my retina changing its
physical characteristics, and my inner seeing of the car keys on
the mantel would not be instrumental in my memory. Images would
not be made, used, or invoked by me, but would be the results of
other, purely physical, causes. We could then debate about
whether these other causes should themselves be called images,
descriptions, or some combination of them, but this would not
change the fact that my imagery experiences would be left out of
the causal loop.
Although only two theories were examined, I believe the
epiphenomenal status of imagery in computational systems is
really quite general: computationalism results in
epiphenomenalism for images because images are non-computable
items. Some theorists are happy with this state of affairs for
images (Pylyshyn, also see Jackendoff, 1989) because they see all
our behavioral capabilities compatible with computationalism.
Others, like Kosslyn and Tye, have attempted to rescue our
intuition that our imagery experiences are causally related to
behavior, and have also tried to simultaneously maintain a
computational view of the mind. The result is forced analogies
between computational states and conscious states or the false
argument that whatever is designated to be functioning as an
image in a computer is itself an image. Experimental evidence in
this vein is little better, since it ultimately relies on
intuitions derived from our experiences rather than the
requirements of computationalism. Epiphenomenalism is not
eradicated in these attempts, just covered over.
These criticisms of epiphenomenalism do not apply to a
somewhat looser form of epiphenomenalism that might be appealed
to as falling under the general rubric of cognitive psychology
(as opposed to the computational physicalism we have been
considering here). This form of epiphenomenalism is one that can
be argued for, and is quite compatible, given a correct
understanding, with philosophical imagism. The argument is in
two steps. The first is to employ notions of "cognition" and
"judgment" in the unconscious. Imagine a system cognitive
system that accepts inputs from perception or the imagination,
processes this data, and then yields a result in the form of a
new cognitive state. Let us call this new state a knowledge
state or a judgment state. Suppose also that this process
happens unconsciously without the intrusion of conscious
rationality. It seems reasonable to assert that there are
cognitive steps involved in the process even if we are not
conscious of them.
This form of identifying what is a cognitive step makes no
reference to possible reduction, and it does not specify whether
the causes of the intervening steps or the steps themselves are
physical, mental, or if they conform to demands of
computationalism. Unconscious cognition, in this sense of the
term, occurs all the time and on many levels. I previously
pointed out in the case of color constancy, but one can add
multitudinous examples at will: adding numbers without being
aware of every step, judging that an object will fall before it
falls, recognizing a picture of a familiar face immediately, and
so on. (I argued in chapter 3 that such examples are not do not
involve "images," however. The notion of an unconscious image,
as Pylyshyn has repeatedly argued, has little to recommend it.)
The next step is to add consciousness and a phenomenal image
to the system, as indicated below.
input ---> unconscious ----> phenomenal image ----> judgment
processing ^
| |
|---->----cognitive flow (?)------>----|
In this system, the unconscious processing produces a phenomenal
mental image, but (it can be asserted) the actual cognitive flow
remains just as it was before the image was introduced: the
cognitive flow is around, rather than through the image.
Therefore, the conscious mental image is an epiphenomenal result
of cognitive processing. As Pylyshyn has argued many times,
support for this model follows from elementary theoretical and
empirical considerations. Recall Pylyshyn's argument on the use
of images in memorization tasks (from chapter 4): If I use the
image of a dog riding a bicycle to remember that the forth item
on the list I wish to memorize is a bicycle, it can not be the
image of the bicycle that brings about the recall. The image
might as easily have designated "riding" or "amusement" as
"bicycle." Therefore, he concludes, it is not the image that is
responsible for generating the judgment that the forth item on
the memorized list was "bicycle." Since the phenomenal image
falls out of the flow of the actual cognitive processing it is
epiphenomenal. This phenomenon of memory is well studied and
even Kosslyn agrees that in these cases phenomenal imagery is
sometimes epiphenomenal.
There is one sense in which there is nothing wrong with this
argument. This is, in fact, based one of the fundamental
descriptivist complaints against imagism listed in chapter 2: the
no content objection. I accepted this argument as valid, but
only if images are considered to be raw appearances with no
intentional content. The problem is that mental images generally
resist, on both phenomenological and empirical grounds, the
reduction to raw visual data. Therefore, this simple account of
the relation between cognition, judgment and the phenomenal image
always needs to be filled in.
Philosophical imagism points toward the fact that how and
with what we reach our judgments, when imagery is present in
consciousness, is an ambiguous and difficult matter to start
with. Imagery experiences, as I have argued, are nearly always
involve being concerned with both the visual and the intentional
content. We inwardly "see" a dog on a bicycle, but we remember
that the forth item on the list was "bicycle." The
descriptivist-inspired argument that because these forms of
content can be distinguished, they must be separate in mental
causation (i.e., as we experience it, rather than as it is
defined by some external standard of what cognitive steps are),
seems to me to be a dubious one.
Memory is much more complicated than such an example makes
out. On the one hand, it often seems that the phenomenal image
is entirely necessary. What if the subject does not recall the
forth item on the list because he simply can not conjure up any
image? In that case, the absence of the phenomenal image blocks
the recall of the desired item. We can also take into
consideration a counterfactual claim: if the presence of the
image would have resulted in the recall, then having the
conscious phenomenal image was necessary in the causal chain and
therefore not epiphenomenal. On the other hand, memory does not
always work this way. The phenomenal image is sometimes
superfluous. A subject may remember the item in question despite
being unable to form an image.
Now, none of these considerations results in conclusive
proof about the causal status of phenomenal image in any given
instance. The point for philosophical imagism is not to secure
(univocally) the status of the visual component of imagery, but
to point to what having an image is like in the first place.
There will always be cases in memory that seem clearly to
indicate (practical) necessity, those that do not, and those that
are ambiguous. Similar points about the active and passive,
conceptual and visual aspects of any sort of image may be made,
as in chapter 3. Phenomenologically, we can always point to some
example of an image (e.g., one I might have while thinking or
speaking about any given topic that seems utterly extraneous to
the topic) that is epiphenomenal with respect to some other
process (conscious or unconscious) that may be occurring. What
matters is that there are some instances where we have every
reason to believe that there are cases where an image can be an
efficient cause (causing shock, surprise, or emotion) or an
instrumental cause (in imagination or thought).
The difference between philosophical imagism and the
accounts examined here is that once "cognition" is reconstrued so
that it loses its neutrality and becomes determined by
computational constraints and the notion of physical causality,
the phenomenal image becomes epiphenomenal in every case.
C. Computer User Dualism
At the beginning of chapter 4 (end of "Epiphenomenalism
section), I mentioned the incipient dualism of computationalism.
In an attempt to rescue our intuitions, computational imagists
often revert to what I call computer-user dualism. In attempting
to bridge the gap between saying
I inspect an inner image
and
There is an inner computational array that is the
functional image (constituted by physical brain states)
in the human system,
imagist theorists sometimes revert to phraseology in which the
subject is said to directly inspect arrays or other computational
structures. For example, Tye's
"...I need only look initially at [brain!] representations
of the shapes of general classes..." (Tye, 1991, pp. 82-83).
The phrase is unintelligible from the common sense point of view.
Humans never report directly inspecting brain states. The
theory-laden meaning of the phrase, that what "I" am doing when I
search for a memory image is really to sort through inner
computational representations, is just dualism in disguise. In a
consistent computational system there can be no self-conscious
agents standing outside a computational system. Conscious,
agent-controlled activity is an illusion produced (miraculously)
by the system itself. It can not be "I" in the ordinary sense
of the word -- who sorts through computational arrays; it can
only be that some blind series of physical events are occurring
in which "I" again, in the ordinary sense of self-conscious
awareness never appear. How then, does the physical process
restore the lost subjectivity? In such phrases as this, the
answer is "by implication and through the equivocal use of the
subject term," that is, by saying that although the process is
totally blind, "I" nevertheless perform the process. Because
both senses of "I," the ordinary subjective sense, and the
mechanical system sense are used, the intelligibility of such
phrases rests on the hidden presupposition that human subjects,
with their full conscious faculties of reason and visual
discrimination are, in fact, users of an entirely separate
mechanical system, devoid of semantic content, that yields up to
them various requested forms of information. The conscious "I"
appears to search for memory information just by willing it, this
activates a computer mechanism that does the unconscious search
through "information" that is devoid of semantic content, and
then the result is returned to consciousness. In short, the
system is no different than Descartes's scheme in which
information was displayed to the conscious agent on an inner
screen.
At the end of Chapter 3, I suggested that if the brain were
an information processing machine, there should be no reason that
its principles of operation would not be discoverable. If it
were an information processing machine, we could get past the
suggestive metaphor of saying that we have or process information
and find the actual mechanisms, the actual information, and the
actual processes that do this. I think now, after our review of
two sample information processing systems, it should be clear
that getting past the metaphor is difficult if not impossible.
There are some reasons for this that the metaphysician of
consciousness might have put forward at the beginning, and
perhaps obviated the entire discussion. Computational and
mechanical systems are systems that move irrevocably forward due
to events of the past; they have no way to globally synthesize
the present in conscious awareness of the past and the
possibilities of the future, and no principle of self-directed
activity arising in a spontaneous (mental) act that has no prior
history in "physical" (17th Century sense) terms. In mechanical
systems, there are no relative beings or potential beings;
everything is actual. There are no representations in computers
because there is nothing in a computer to whom anything could be
represented. These limitations remove the basis for metaphysical
possibility of self-conscious awareness. As a result, there are
no conscious seeing activities and no images in such systems. To
the extent any such systems might be instantiated in the brain,
they will terminate in their explanatory history of computational
events at a stage in which neurophysiological facts take over and
then pronouncement is made: "and then we see" or "and then we
have an image" which is just to say that the mind/body problem
has not been solved, we are exactly where we started and we will
be left with the appeal to the forms of explanation we already
visited, i.e., ones that combine an analysis of intentional
awareness with neurophysiological facts.
It seems to me, then, that computationalism is likely to end
(consistently) in epiphenomenalism or (inconsistently) in
computer-user dualism. So, if computationalism implies
epiphenomenalism, and epiphenomenalism is not true,
computationalism can not be a true theory of the mind. As far as
images are concerned, these systems turn the image into an object
in the wrong senses we sought to avoid. It is either an
epiphenomenal (conscious) object or a physical (unconscious)
brain state only. The idea that mental images can be constituted
in and through acts of the subject is lost.
Figure 5-1. Shepard and Metzler Figures
Figure 5-2. Shepard and Metzler's Results
Figure 5-3. Average Improvement in Response Time for First Three
Attempts at Shepard and Metzler Task. (Note that all attempts
except those at 140 and 160 degrees show first try is the
slowest. Number of students tested = 10.)
Figure 5-4. Successive Tests of 140 Pairs of Shepard and Metzler
Figures by a Single Subject. (Upper curve is first test, lower
curve is second test.)
Figure 5-5. Kosslyn's Map
Figure 5-6. Map Scanning Results