Physicist Richard Feynman used a metaphor of "Cargo Cult Science" in a 1974 Caltech commencement address to criticize an attitude that he felt pervades the the social sciences, and psychology in particular:
I think the educational and psychological studies I mentioned are examples of what I would like to call Cargo Cult Science. In the South Seas there is a Cargo Cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they’ve arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas—he’s the controller—and they wait for the airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things Cargo Cult Science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land.
Now it behooves me, of course, to tell you what they’re missing. But it would he just about as difficult to explain to the South Sea Islanders how they have to arrange things so that they get some wealth in their system. It is not something simple like telling them how to improve the shapes of the earphones. But there is one feature I notice that is generally missing in Cargo Cult Science. That is the idea that we all hope you have learned in studying science in school—we never explicitly say what this is, but just hope that you catch on by all the examples of scientific investigation. It is interesting, therefore, to bring it out now and speak of it explicitly. It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty—a kind of leaning over backwards. For example, if you’re doing an experiment, you should report everything that you think might make it invalid—not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment, and how they worked—to make sure the other fellow can tell they have been eliminated.
Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can—if you know anything at all wrong, or possibly wrong—to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it. There is also a more subtle problem. When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition.
The close relationship of neuroscience and the social sciences causes this problem to spill over when "neuro-[x]" studies are done. Neuroscientists today often say they are trying to tease out "functions", a set of constructs usually defined in terms of psychology, cognitive science, philosophy, or sociology concepts presumed to have evolutionary functions. When they are not found in the brain as modules, but rather the brain uses different kinds of modular structure, the brain is denounced for exhibiting 'neural reuse' or 'neural degeneracy' with respect to these posited functions, that must somehow be "emergent" from the brain.
Feynman closes by alluding to the strong conformance pressures that are applied to social science researchers, and wishing Caltech students an environment free of them:
So I wish to you—I have no more time, so I have just one wish for you—the good luck to be somewhere where you are free to maintain the kind of integrity I have described, and where you do not feel forced by a need to maintain your position in the organization, or financial support, or so on, to lose your integrity. May you have that freedom.
Such freedom is even harder to achieve in the present neuropsychology research ecosystem than in Feynman's day, with grant reviewers, ethicists, IRBs, peer reviewers, administrators, supervisors, and journalists all expecting research to use the rubric of extant mental process constructs. Neuromythography is an attempt to reform this by providing a de novo bottom up approach by coalescing neuroscience itself with artful metaphor instead of philosophy.
Here's a link to the whole Feynman speech.
