admin | 
11 17th, 2008| 
During the 1970s, social psychology was of two minds when it came to experimentation. As shown in Figures 1 through 3, early in the decade there was a marked increase in experimentation relative to individual differences approaches. Even as researchers turned increasingly to experimental approaches, however, the field underwent a wrenching “crisis” that was marked by considerable doubts regarding the utility of experimentation (see Jones, 1998, for a discussion). Questions arose regarding both the internal as well as external validity of experiments. Together, these questions may have diminished experimentation’s appeal in much the same way that Mischel’s critique sucked the wind from personality psychology’s sails several years earlier. Difficulties in Establishing the Internal Validity of Experiments In theory, experiments can provide higher levels of confidence regarding the causal mechanisms underlying phenomena than correlational approaches. What the crisis forced social psychologists to realize, how- ever, was that the results of experiments are not always as illuminating as they could be in principle. Although not among the instigators of the crisis, Aronson and Carlsmith (1968) articulated a central concern, How can we be sure that this operation is, in fact, an empirical realization of our conceptual variable? Or conversely, how can we abstract a conceptual variable from our procedures? . . . There is no cheap solution to the problem. This is largely due to the fact that in social psychology there exist relatively few standard methods of manipulating any given conceptual variable.
(pp. 14-15) But the problem goes much deeper than this. Consider the common assumption that one advantage of experimental designs over nonexperimental designs is that, through randomization, researchers eliminate the “omitted variable” problem (i.e., the possibility that some variable other than the one that was manipulated was responsible for an effect). This is not true of social psychological experimentation or, for that matter, any other type of experimentation. Bollen (1980) notes that “in any experiment we must be aware that variables other than the intended one may be influenced by the treatment
and that these other variables could be responsible for the effects found” (p. 76). In theory, the omitted variable problem could be addressed by a procedure for validating manipulations— parallel to the construct validation process designed by Cronbach and Meehl (1954) for psychological tests.
Although Brunswik and his students (e.g., 1944, 1947; for a overview, see Hammond, 1980) attempted to address this issue through the development of his representative design” and “rich stimulus” approaches, these approaches have garnered little support by social psychologists. Perhaps the most serious effort to address the validity issue has been the use of “manipulation checks.” Nevertheless, on those infrequent occasions wherein manipulation checks are used and reported, they are themselves of unknown validity. Moreover, if the results of manipulation checks fail to converge with the primary findings of the study, investigators often assume that the manipulation check was faulty or that the effects of the manipulation were simply inaccessible to participants (e.g., Nisbett & Wilson, 1977). And even if valid manipulation checks were used routinely, they would not fully address the omitted variable problem because they are only capable of revealing what the manipulation did do; they cannot provide assurances that the manipulations’ effects were limited to those intended.
Although the omitted variable problem is not fatal, recognition of this and related problems did dampen some of the enthusiasm enjoyed by advocates of experimental approaches. This problem was further compounded by the recognition of experimentation’s problems with external validity. Read the rest of this entry »