In his 1985 book Surely You're Joking Mr. Feynman, Richard Feynman describes what he calls 'Cargo Cult' Science:
In the South Seas there is a cargo cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they've arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas - he's the controller - and they wait for the airplanes to land. They're doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn't work. No airplanes land. So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they're missing something essential, because the planes don't land. (p. 310 - 311)
I often worry that academic psychology is becoming a Cargo Cult Science. Psychologists have mastered the arts of experimental design and statistical inference. They do everything right. The form is perfect. But I don't see many airplanes landing. That is, I see lots of publications of clever paradigmatic experiments, but have difficulty extracting much value from this literature for understanding human experience, particularly in the context of complex work - such as clinical medicine. This vast scientific literature does not seem to generalize in ways that suggest practical ways to improve the quality of human experience.
On the surface, these papers appear to be addressing practical issues associated with cognition (e.g., decision making, trust, team work, etc.), but when I dig a bit deeper I am often disappointed, finding that these phenomenon have been trivialized in ways that make it impossible for me to recognize anything that aligns with my life experiences. Thus, I become quite skeptical that the experiments will generalize in any interesting way to more natural contexts. Often the experiments are clever variations on previous research. The experimental designs provide tight control over variables and minimize confounds. The statistical models are often quite elegant. Yet, ultimately the questions asked are simply uninteresting with no obvious implications for practical applications.
Not everyone seems to be caught in this cult. However, those that choose to explore human performance in more natural settings that are more representative of the realities of everyday cognition are often marginalized within the academy and their work is typically dismissed as applied. For all practical purposes, when an academic psychologist says 'applied science' s/he generally means 'not science at all.'
Perhaps, I have simply gotten old and cynical. But I worry that in the pursuit of getting the form of the experiments to be perfect, the academic field of psychology may have lost sight of the phenomenon of human experience.
Much of what JF says is true; But it IS worth while looking at, citing, and publicizing the exceptions to this lack of real world applications.
Also bearing in mind Jack Adams findings of a typical 17 year lag between applied research in HF, and its applications, particularly in safety-critical systems.
Also, it is important to note that the work does not have to be motivated by a desire for application. Work motivated by pure curiosity can often turn out to have important practical implications that the original researchers did not anticipate. The key is that the research be well grounded with respect to natural human experience. Most experimental paradigms were originally motivated by interest in natural phenomenon. However, sometimes the paradigms take on a life of their own. For example, the focus shifts from trying to understand perceptual motor control or attention and people begin studying Fitts' Law or the Sternberg Task. The result is often elegant experiments that provide little insight beyond the original work.
Pingback: Reproducibility Crisis in Psychology | Perspicacity