Category: Uncategorized

This is a follow up to earlier posts raising questions about the experience of time and exploring some of the ontological and epistemological implications.  Following some responses to the earlier post, I went back to a source that has had a significant impact on how I think about time and about the ontological and epistemological bases for cognitive science: James Gibson.  And I came to the conclusion that I could not say it better than Gibson already did. So, I would like to share some quotes from Gibson (1979) on the nature of space and time:

I am ... asking the reader to suppose that the concept of space has nothing to do with perception. Geometrical space is a pure abstraction. Outer space can be visualized but cannot be seen. The cues for depth refer to paintings, nothing more. The visual third dimension is a misapplication of Descartes's notion of three axes for a coordinate system.

The doctrine that we could not perceive the world around us unless we already had the concept of space is nonsense. It is quite the other way around: We could not conceive of empty space unless we could see the ground under our feet and the sky above. Space is a myth, a ghost, a fiction for geometers. All that sounds very  strange, no doubt but I urge the reader to entertain the hypothesis. For if you agree to abandon the dogma that "percepts without concepts are blind," as Kant put it, a deep theoretical mess, a genuine quagmire, will dry up. (p. 3)

We live in an environment consisting of substances that are more or less substantial, of a medium, the gaseous atmosphere; and of the surfaces that separate the substances from the medium. We do not live in "space." (p. 32)

The world of physical reality does not consist of meaningful things. The world of ecological reality, as I have been trying to describe it, does. If what we perceived were the entities of physics and mathematics, meaning would have to be imposed on them. But if what we perceive are the entities of environmental science, their meanings can be discovered. (p. 33)

Events as the Primary Realities. In the first place, the flow of ecological events is distinct from the abstract passage of time assumed in physics. The stream of events is heterogeneous and differentiated into parts, whereas the passage of time is supposed to be homogeneous and linear. Isaac Newton asserted that "absolute, true, and mathematical time, of itself and from its own nature, flows equably without relation to anything external." But this is a convenient myth. It assumes that events occur "in" time and that time is empty unlessed "filled." This habitual way of thinking puts the cart before the horse. We should begin thinking of events as the primary realities and of time as an abstraction from them - a concept derived mainly from regular repeating events, such as the ticking of clocks. Events are perceived, but time is not.

It is the same with space as with time. Objects do not fill space, for there was no such thing as empty space to begin with. The persisting surfaces of the environment are what provide the framework for reality. The world was never a void. As for the medium, the region in which motion and locomotion occur, wher light can reverberate and surfaces can be illuminated, this might be called a room but it is not space. Surfaces and their layout are perceived, but space is not, as I have long been arguing. 

It might be said, without going as far as I have done above, that time consists of the events filling it and that space consists of the objects filling it. But I will argue that this formula still perpetuates the fallacy. The metaphor of filling is wrong. Time and space are not empty receptacles to be filled,; instead, they are simply the ghosts of events and surfaces. 

Time is not another dimension of space, a fourth dimension, as modern physics assumes for reasons of mathematical convenience. The reality underlying the dimension of time is the sequential order of events, and the reality underlying the dimensions of space is the adjacent order of objects or surface parts. Sequential order is not comparable to adjacent order; it is not even analogous to adjacent order. For the order of events cannot be permuted, whereas the order of parts can. You can reshuffle the parts but not the events, as you can rearrange the furniture in a room but not the happenings that occur in it.  (p. 100 - 101). 

I hope this sampling of quotes will give a taste of Gibson's perspective and that it will lead people to read or re-read his important book: "The Ecological Approach to Visual Perception." 

To close, I would like to briefly comment on the ontological implications of these ideas. The Newtonian view of the world is that space and time are critical elements of reality. In essence they are the frame or coordinate system against which objects, motions, and other things are measured. Though this idea is now questioned by physicists - this idea that there is a coordinate system or box which is a fundamental aspect of reality that stands outside or independent from our experiences is generally accepted by many social scientists, Thus, psychologists study space and time perception as fundamental to theories of human cognition. And they also assume that somehow this external coordinate system has to be reconstructed as mental models in order for us to make sense of the world.

This leads to a dualistic ontology. Since, time and space can't be experienced directly - they constitute an "objective" reality that is independent from our "subjective" experiences. Thus, our ontology need two circles - one for the physical world of space and time - more generally referred to as the reality of physics or of matter; and another for the experiences of surfaces and events that is more generally referred to as mind. 

However, if Gibson is right, that time and space are 'ghosts' or 'fictions' (created to make the mathematics tractable), then there is no reason to posit a reality that is independent from our experience. If Gibson is correct then the idea of an independent "objective" reality is a myth. And once we dispense with the myth of an objective world apart from our experiences - we are left with the single reality of Experience (James) or Quality (Pirsig).  

The dominance of dualism in Western culture is reflected in the fact that the discussion of reality was framed around two separate circles - one for mind and the other for matter. However, there is another possible framing of the question. Consider the possibility that there is only a single reality which is a joint function of BOTH mind AND matter.

 

This is the position that was preferred by the American Philosopher and Psychologist, William James. He called this single reality that was BOTH subjective AND objective “experience.” Robert Pirsig, author of Zen and the Art of Motorcycle Maintenance and Lila, came to a very similar conclusion. He called the joint function of mind and matter “Quality.” We have come to believe that this quality or experience is “What Matters” when it comes to reality.

An important implication of this alternative framing of the question of reality is that our experiences are real. That although two people might experience the same event differently. Their experiences are real. The experiences are more real than the ticks on the clock or the ticks on a ruler that we use to map them.

For this position to make sense, it is necessary to accept that the various experiences of different people are all real. Rather than dismissing the differences as subjective biases, consider that the differences are differences of perspective or context. Consider that reality and experience are more than a collection of isolated points in time, but rather they are trajectories flowing over time. And thus, the same point will be experienced differently depending on the larger context – the trajectory over time (or life-line) that it is part of.

This also requires that we reconsider how we frame questions of time. Rather than thinking about the ticks of the clock as a reality that is extrinsic to our experiences, we might start with the idea of motion or flow of activity as an intrinsic property of our experience. Clocks are useful tools for synchronizing our motions with the people and objects around us – but the flow is the reality. The flow is the context that grounds events and moments of experience in reality.

Conventionally, the question, “What is reality?” is framed as an EITHER/OR proposition that is illustrated as a Venn diagram. Within this framework, the options seem to be - to either accept one of the circles as real, and the other as an illusion, or to accept both circles as two different, distinct realities.

Accepting mind as the single reality is called mentalism. While this is generally presented as a logical possibility, few people give serious consideration to this possibility. In essence, this suggests that we are all living in the Matrix – that there is no matter – it is purely a construction of our minds.

Accepting matter as the single reality is called materialism. This position is embraced by many people. The implication of this position is that the mind is a by-product of matter and that the experiences that we attribute to mind are nothing more than the results of chemical and biological processes in brains.

The final option, that tends to dominate Western culture is that there are two distinct realities. On the one hand, there is the objective world of matter – that is the domain of the physical and biological sciences. And on the other hand, there is the subjective world of mind – that is the domain of the social sciences and religion. This is called dualism, and in contrast to materialism, dualism contends that there are real aspects of our mental and spiritual lives that cannot be completely reduced to chemical and biological brain processes.

Dualism, however, raises another puzzle. If mind and matter are two distinct realities, how is it possible for them to interact? How is it possible for a mind to know the world of matter?

A large body of research suggests that the mind has great difficulty in coming to know the world of matter as it is. The research on human perception and cognition suggests that the mind is prone to illusions and biases. Conventional wisdom suggests that we construct mental models of the actual world using partial, noisy data from our senses. For example, we must construct a model of 3-dimensional space from two distorted, upside-down, 2-dimensional retinal images, augmented by our tactile senses.  

It's not surprising that our experiences don’t always correspond with the world of objective reality associated with the ticks of a clock or the ticks on a ruler. Actually, the surprising thing is that sometimes people are able to interact with the world quite skillfully – to run through the forest over rough terrain without colliding with a tree; to drive at highway speeds in dense traffic without colliding with another car; to pitch a baseball accurately, to hit it, and to catch a fly ball; to intercept and volley a soccer ball past a goalie; to fly and land an aircraft safely.

 

 

Conventionally, the question, “What is reality?” is framed as an EITHER/OR proposition that is illustrated as a Venn diagram. Within this framework, the options seem to be - to either accept one of the circles as real, and the other as an illusion, or to accept both circles as two different, distinct realities.

Accepting mind as the single reality is called mentalism. While this is generally presented as a logical possibility, few people give serious consideration to this possibility. In essence, this suggests that we are all living in the Matrix – that there is no matter – it is purely a construction of our minds.

Accepting matter as the single reality is called materialism. This position is embraced by many people. The implication of this position is that the mind is a by-product of matter and that the experiences that we attribute to mind are nothing more than the results of chemical and biological processes in brains.

The final option, that tends to dominate Western culture is that there are two distinct realities. On the one hand, there is the objective world of matter – that is the domain of the physical and biological sciences. And on the other hand, there is the subjective world of mind – that is the domain of the social sciences and religion. This is called dualism, and in contrast to materialism, dualism contends that there are real aspects of our mental and spiritual lives that cannot be completely reduced to chemical and biological brain processes.

Dualism, however, raises another puzzle. If mind and matter are two distinct realities, how is it possible for them to interact? How is it possible for a mind to know the world of matter?

A large body of research suggests that the mind has great difficulty in coming to know the world of matter as it is. The research on human perception and cognition suggests that the mind is prone to illusions and biases. Conventional wisdom suggests that we construct mental models of the actual world using partial, noisy data from our senses. For example, we must construct a model of 3-dimensional space from two distorted, upside-down, 2-dimensional retinal images, augmented by our tactile senses.  

It's not surprising that our experiences don’t always correspond with the world of objective reality associated with the ticks of a clock or the ticks on a ruler. Actually, the surprising thing is that sometimes people are able to interact with the world quite skillfully – to run through the forest over rough terrain without colliding with a tree; to drive at highway speeds in dense traffic without colliding with another car; to pitch a baseball accurately, to hit it, and to catch a fly ball; to intercept and volley a soccer ball past a goalie; to fly and land an aircraft safely.

 

 

 

 

Have you ever become so engrossed in an event that time just flew by? When an hour seemed like minutes? Have you ever been surprised to find that a companion did not share your feelings? That the event that you found to be intensely interesting, she found to be mind-numbingly boring? What to you seemed like moments, seemed to her to be an eternity?

Whose impression was right? What was the real time?

When people experience the same event differently – the differences are said to be subjective biases. The people are not being objective. Obviously, the real time is that measured by the clock. And any experiences that don’t correspond with what the clock specifies must be biased or illusionary.

When our experiences of time don’t correspond with the ticks on a clock; or when our experiences of distance don’t correspond with the ticks on a ruler, conventional wisdom suggests that we have made a mistake, that we are in error – that our experiences are illusions.

Obviously, all the different impressions of the duration of an event by a diverse group of people can’t be real. Only one impression can be right – and that is the impression that is consistent with the objective measurement made with a clock. An hour is an hour. And if the experience does not agree with the ticks of the clock, the experience must be wrong.

I wonder – did time exist before there were clocks?

Clocks and rulers are wonderful inventions. Together they help us to map out our world and to coordinate and synchronize our actions with others. They allow us to plan our days and map out our world – to schedule when and where to join others for work, to share a meal, or to play.

Yes, clocks and rulers are wonderful constructions, but they are constructions non-the-less. They are constructions that were designed specifically to objectify the world, to eliminate the subjective dimensions of experience from the equation.

Conventional wisdom accepts the measures of clocks and rulers as the objective reality. These are some of the tools we use to describe the objective world of matter. But what about peoples’ experiences? If they are not consistent with the objective world of matter – are they real?

Conventional wisdom suggests that there are two choices: EITHER our experiences are an illusionary by-product of matter; OR they may be part of a separate, different reality that we call mind.

But what if conventional wisdom is wrong? What if we have mistaken the map that we constructed with our clocks and rulers as reality? What if our experiences are actually more real than the ticks on the clock?

Are there alternatives to ticks on the clock for thinking about events, durations, and synchronicity? One alternative is the frequency domain - that is characterizing patterns over time as composed of sums of sine waves. For example, insights into the performance of many systems can be gained from examining the frequency response. Many systems exhibit 1/f characteristics and stability of control systems can be inferred from Bode plots that show the frequency response. 

Another framework is to take events as the objective reality - and then frame time in event relative terms reflecting timing and windows of opportunity - too soon, too late, too fast, too slow, too short, too long ... 

Clocks are useful for coordinating social activities - but maybe we should consider alternative ways to think about how events are experienced by biological systems. 

When I first switched majors to Psychology, I was intending to be a social worker or counselor. However, as I got deeper into Psychology, I was intrigued by the ability to model human information processing using information statistics and control theory (e.g., HIck-Hyman Law, Fitts Law, McRuer & Jex's Crossover Model, Signal Detection Theory). I began to think of myself as a scientist, rather than a practitioner. As a professor of Psychology, I taught undergraduate courses in experimental methods and supervised graduate research. But if I am honest with myself, I have to admit that I never quite lived up to the standards of science that I set for the students. 

The fact is, that when my intuitions conflicted with the conventional wisdom from experimental psychology, I tended to trust my intuitions, rather than the conventional wisdom. And I trusted my intuitions, even when I couldn't put those intuitions into convincing arguments. For example, very early in my career the Ecological Psychology approach of James Gibson and constructs such as affordance, optical invariant, direct perception, and attunement seemed to match my intuitions. But I wasn't able to articulate why these constructs made sense or to defend these views against the conventional wisdom of the day. This is one of the reasons I failed to get tenure in my first academic job.

When I designed experiments, I wasn't testing hypotheses, rather I was attempting to demonstrate the plausibility of my intuitions to those who were skeptical. There was a certain degree of blind faith in my intuitions - a clear confirmation bias that shaped my interpretations of both my experimental research and the literature. Looking back, it is clear that my goal was not to test my intuitions - but to defend them. 

You would think that more than 30 years as a researcher would gradually tame this tendency to trust my intuitions. You would think that I would eventually learn how to think like a scientist. However, the opposite is true. My faith in my intuitions is stronger today, than ever! This is partly due to the fact that the conventional wisdom has changed to incorporate some of my early intuitions. For example, Gibson's ideas about optical specification of relative motion have inspired innovations for robotic vision and the construct of affordance has proven to be very useful for designers. 

I have also improved my ability to give words to my intuitions. More than 30 years of teaching has helped me to select the right words, right metaphors, and right images for framing these intuitions so they don't sound quite as crazy or naive, as they first did. 

Today, I rationalize this faith in intuition as being based on many years of living. I convince myself that my personal experiences are data relevant to a science of psychology. Yes, it is a limited perspective, but it is deep! Are our everyday experiences valid as scientific data for Psychology?  Or, is this pure rationalization? Should I simply admit, that I am a poor scientist? Perhaps, I should consider changing fields to philosophy. 

 

Thanks to Don Norman for sharing the intro to his new book, in which he comments on the pervasive impact of design and  suggests that the impact may not always lead to more satisfying, high-quality experiences. He goes on to suggest that there is a need to change how we teach design. While I agree with Don, I believe that the problems are not specific to Design Education. I would argue that the problems are pervasive across the board, when it comes to university education. 

Current educational systems are organized around disciplines and techniques. The emphasis on HOW has left little room for consideration and discussion of WHY.  The emphasis on technical skills has left little room for consideration and discussion of philosophy, little room for explorations of fundamental questions about the nature of experience.  For example, I have been shocked to discover that many students trained in systems engineering are clueless about general systems thinking. And I have long regretted the emphasis in the cognitive sciences on experimental and modeling methods at the expense of considerations of the philosophical roots of the science.

Current educational systems have been designed around selling answers and solutions - rather than around exploring questions. The educational systems have been organized around authority based on POWER and POLITICS, rather than authority based on CURIOSITY and EXPERIENCE. We need new educational systems designed around LEARNING, rather than around TEACHING. 

Yes, Don Norman is right, but I hope people don't place the blame exclusively on Design Education and I hope they don't look exclusively to Design Education for the solution. Perhaps, Designers are the people who are best suited to lead the revolution. But the problems Don describes will ultimately require a more pervasive change to universities. It will require us to tear down the walls that separate disciplines and it will require educators, in general, to be more humble with respect to the current answers that they are selling. It will require educators to listen more carefully to the questions their students are asking. It will require re-envisioning education to be a process of co-exploration across generational perspectives - rather than a process of passing answers and techniques from one generation to the next. 

Four years ago, I left academia to join a small design startup, Mile Two. Ironically, after more than 30 years in academia teaching about applied cognitive psychology, human factors and cognitive systems engineering, it feels like my education on design has just begun. I discovered that essential people on our design teams were trained very differently than I was and brought skills and talents to the team that I was lacking. I recently collaborated with some of these teammates on a chapter to reflect on some of the different perspectives that contribute to the development of software to support cognitive work. The first figure was our attempt to summarize some of the different disciplinary perspectives and the second figure is a more elegant revision of that figure suggested by my friend and colleague Fred Voorhorst. 

Figure 1. Multiple perspectives on design from Flach, Bennett, Butler & Heroux (In press Handbook of Human Factors). 

These diagrams suggest four classes of constraint that must be considered in the design and development process:

• Agency Constraints: These are properties of the "agents" in the system. Traditionally, the agents have been humans, but as machines becoming increasingly autonomous - we now have to consider the internal constraints of these autonomous agents. 
• Implementation Constraints: These reflect properties of the computers that will host the software. They include things such as the types of displays and the control/input devices.
• Functional Constraints: These are properties of the work domains or the problem spaces that are the targets for the cognitive work. These include the physical, regulatory, and organizational dynamics that limit the range of possible actions and consequences. 
• Aesthetic Constraints: These are properties of the psychological and cultural values that determine the degree of satisfaction or frustration associated with using the software products. 

All of the four disciplines, at least implicitly, recognize the full range of constraints, however, each discipline tends to emphasize a subset of these constraints in training and practice. 

Human Factors [HF] tends to emphasize the constraints associated with human agency. That is, the focus is on specifying the physical [e.g., reach envelops], perceptual [e.g., visual acuity] and cognitive [e.g., working memory capacity] limitations of humans, so that these limitations are respected in the final design. 

User Interface Design [UI] or Human Computer Interaction [HCI] tends to emphasis the impact of various computer capabilities on performance. For example considerations include types of displays [e.g., table-top, large-screen, hand-helds, or head mounted], types of controls [e.g., keyboard, mouse, game controller, touchscreen], and types of mediation [e.g., co-located or distributed via internet]. 

Cognitive Systems Engineering [CSE] focuses on the problem or work domain constraints. This involves describing the means-ends relations in terms of affordances and exploring alternative strategies or heuristics that might be used to efficiently navigate the functional space. 

User-eXperience Design [UX] tends to consider the larger psychological and cultural context of use that impacts people's self-image and sense of satisfaction when using the design product. 

I come from an HF perspective and was involved in the early development of the CSE perspective. Thus, much of my career has been spent arguing for the value of a CSE perspective for filling gaps in the design space that I felt were not adequately addressed by the HF perspective. The point was not that the HF perspective was wrong, but that it was not sufficient. 

However, over the last few years spent developing software products I have come to realize that even the combination of HF and CSE leaves many gaps in the design space that require skills and perspectives associated with UI and UX design. I realize that my sense of the full demands of design were still quite naive and inadequate. 

I now regret that I did not have and was not able to give my students a broader sense of the full breadth of the design challenge. Yet, I realize that it would be impractical to cover the full breadth of design in any reasonable course of study. Thus, much of the education must continue beyond the formal course work. And ultimately, I have come to the conclusion that design is a team sport that requires a diverse set of perspectives and people who are open to and  who welcome alternative perspectives.  To function on such a team I have had to dampen my tendencies to be an advocate for a particular perspective and I have had to heighten my ability to listen to and appreciate the value of alternative perspectives.