CHALLENGING PSEUDOSCIENTIFIC BELIEFS:
SURVEYING EVIDENCE FOR EXOTIC CLAIMS
Richard A. Crowe
Department of Physics and Astronomy
University of Hawaii at Hilo
and
Carole K. Miura
Department of Mathematics
University of Hawaii at Hilo
Summary - We have carried out a preliminary survey to examine what popular beliefs and claims are most readily accepted among our college students and to compare these responses with the results of similar questionnaires taken in other parts of the country. Ratings data for 70 statements dealing with topics in science and pseudoscience were obtained from 330 UH-Hilo college students and 40 UH-Hilo college faculty over a 5-year period. The results are compared to similar statements from four other surveys (Morris, 1981; Miller, 1985; Harrold and Eve, 1986; Gallup and Newport, 1991).
The methodology of science has proven to be a reliable tool in improving our lives. No one can deny this rationally because we all make use of the practical benefits of science every single day. Our society today cannot function without electricity and magnetism, yet all our electrical and communications technology is ultimately derived from the practical application of four mathematical relationships uncovered by early experimenters and then later explained by 19th-century Scottish physicist James Clerk Maxwell (Giancoli, 1985, p. 502). The home entertainment industry and all of our computer technology ultimately derive from the application of the laws of quantum mechanics formulated by theoretical physicists such as Max Planck, Niels Bohr, Paul Dirac, Wolfgang Pauli, Erwin Schroedinger and Werner Heisenberg over the first thirty years of this century (Gribbin, 1984, pp. 141-142). The application of a principle of fluid motion first described by Daniel Bernoulli in the 18th century has allowed us to construct the airplane and other flying machines (Giancoli, 1985, pp. 203-205). The understanding of transfer of thermal energy (heat) developed by 19th-century physicists Rudolf Clausius and Sadi Carnot has given us the internal combustion engine (used in automobiles), the steam engine, and the refrigerator (Giancoli, 1985, pp. 276-279). Advancements in medical knowledge are too numerous to mention here, but suffice to say that the average life expectancy has increased by decades over the last two centuries as a result. The cost of fundamental science is only about 5 % of the cost of applied research and development, yet it contributes greatly to technology, the education of scientists and the enrichment of our culture (Lederman, 1984).
Science is clearly a very powerful endeavor; many have argued that it is the greatest triumph of the human mind. We should all gain a firm grasp of what its potentialities and limitations are, for if we do not, we may suffer grave consequences. Survey after survey has shown that the general public's understanding of science is at an embarrassingly low level (Frazier, 1986a), yet never before in our history have we been so reliant on scientific technology in our everyday lives. Effective science education must show people how to apply the scientific method to the evaluation of exotic claims. If this seems unnecessary to the reader, consider the following sobering examples (see also Randi, 1982, pp. xiii-xv). How are we going to solve an energy crisis if heedless reporters lead the public to believe, without indicating that the claim is undoubtedly false, that perpetual motion machines exist which can derive their energy out of nowhere? How can we get on with the business of dealing with complex legal issues pertaining to the safety and well-being of every citizen when our judicial system is distracted by creationists, who insist, in the face of a large body of interdisciplinary scientific evidence demonstrating conclusively that the Earth is 4.5 billion years old, that school children should be taught alternative young-Earth pseudoscience? In an era when AIDS is a national medical crisis which involves very hard decisions at every level, we are told by holistic practitioners that we can diagnose all our internal medical problems just by inspecting the iris or the spine (two of many widely advertised methods) and that we can solve them with regular intake of special herbal medicines and megadoses of vitamins. How are we going to face up to responsibility for our own actions when we are constantly bombarded by claims that our lives are ruled by the stars or that we need only consult our biorhythm charts? Continued acceptance of such claims by the general public is symptomatic of the poor understanding of the scientific method which is rampant in our society. Even many science educators are prone to ignore the issues at stake, but it is something which we clearly should address before it is too late. What if we are the only intelligent life in the Galaxy (a distinct possibility) and we end up snuffing ourselves out because of our own stupidity, founded upon fear and ignorance? The Earth has a finite number of resources and the problems of energy consumption, pollution, and overpopulation get more complex each year. We cannot afford to waste our time with superstition.
The Survey Questionnaire
Westrum (1976, p. 45) has written ``to encourage the public to believe in science without understanding it should never be the aim of scientists.'' How does the concerned college professor communicate the essence of the scientific method to students? One approach might be to teach them how to evaluate evidence. According to Lett (1990, p. 155), ``we must demand that the evidence for any factual claim be evaluated without self-deception, that it be carefully screened for error, fraud and appropriateness, and that it be substantial and unequivocal.'' Yet how do we design our science courses so that this kind of critical thinking is reinforced? After all, we still have to grade students on their ability to write down clearly the ``right'' answers to questions, whether these be in multiple choice, fill-in-the-blank, true-false, or essay form. How do we teach students to learn how to ask the right questions so that they can solve problems? We need to stimulate such critical thinking in students and do it in such a way so as to maintain an interest in science, but it is a challenging task.
One of us (RAC) has tried to encourage critical thinking by teaching students that examination of the evidence is a crucial element in the methodology of science. To this end, over the past five years at the beginning of each semester (before any instruction was given), he has been administering a survey on the scientific method and exotic claims to General Astronomy students, who are in nearly every case non-science majors. The research goal of our preliminary survey is to examine what popular beliefs and claims are most readily accepted among University of Hawaii at Hilo (UH-Hilo) college students, and to compare these responses with the results of similar questionnaires taken in other parts of the country (Frazier, 1989). The educational goal of the survey is to use the quiz as a tool to inform students about the scientific method, and how scientific inquiry can be applied to all aspects of our culture (not just to questions in the physical sciences). To make the survey legitimate and meaningful, a control group was needed. The most reasonable choice for this was a subset of college professors who are trained in scientific methodology, and who are therefore expected to be more enlightened on issues pertaining to science and pseudoscience. Approximately 40 people on the Faculty of the College of Arts and Sciences at UH-Hilo responded to this survey in 1989.
The survey itself consists of 70 statements taken from a wide variety of topics in science and pseudoscience, and can be found in Crowe &Crowe (1994). The respondents were asked to evaluate their acceptance of each statement on the basis of whether the statement is almost certainly true (category 6), probably true (category 5), possibly true (category 4), questionable (category 3), unlikely to be true (category 2), or almost certainly false (category 1). After the survey was administered to the students, RAC delivered a lecture on scientific methodology, based on the FiLCHeRS (Falsifiability, Logic, Comprehensiveness, Honesty, Replicability, Sufficiency) philosophy of Lett (1990). Subsequently, the results of the survey and the ``best'' responses according to current research as perceived by RAC (see Crowe &Crowe, 1994), along with short explanations and references for further reading, were provided to all student (and faculty) respondents. In Table 1, the statistical results of the survey over the five-year period are shown. The mean response is indicated by ``Avg Ans'' and the standard deviation of the mean is indicated by ``Sigma Mean''. The percentage of the sample which responded in categories 1 or 2 (almost certainly false or probably false) is indicated by % SD or % MD. The percentage of the sample which responded in categories 5 or 6 (probably true or almost certainly true) is indicated by % MA or % SA. The percentage of the sample which responded in category 3 (unsure, but skeptical) is indicated by % US, while the percentage of the sample which responded in category 4 (unsure, but credulous) is indicated by % UC. A standard chi-squared test was performed in order to compare student and faculty responses. The binning procedure was carried out as follows: with 5 degrees of freedom (df), the sample was divided into six groups (1 to 6); with 2 df, the sample was divided into three groups (1-2, 3-4, and 5-6); with 1 df, the sample was divided into two groups (1-2-3, 4-5-6); and with 0 df, the chi-squared test was not valid (expected value less than 5). Expected values for the chi-squared test were calculated by using the faculty members as the ``control'' group. The test was carried out twice for the highest df producing a valid comparison (usually 5, shown in Table 1 as df). The significance level for the test is shown in Table 1 as p. In practice, a value for p of 0.001-0.01 implies that the difference between student and faculty responses is highly significant, since this means a confidence level of better than 99 % . If there was no significant difference between student and faculty responses, then ``Agree'' is indicated.
From the statistics of the data, some trends appear to exist, as evidenced
by clustering of similar responses within categories. These trends (numbered
below as 1-10) are
presented according to decreasing degrees of confidence.
Trend 1: Students are significantly more credulous of psychic
phenomena than are faculty (see
Items 14, 38, 61, and 68, under
Statement Category #3, Psychic Phenomena, in Table 1).
Trend 2: Students have stronger
Bible-based beliefs than do faculty (see Items 7, 23, 47, and 53,
under Statement Category #9, Bible-based Beliefs, and Item 31 under
Statement Category #10, General Science, in Table 1).
Trend 3: Students are more
likely than faculty to accept the printed word as
``truth'' (see Item 10, under Statement Category #7, Consumer Awareness, in Table 1).
Trend 4: Students are less knowledgeable about
physics and astronomy than are faculty (see Items 6, 16, 49, 58, 64, and 67,
under Statement Categories #11-12, Physics and Astronomy, in Table 1).
Trend 5: Students and faculty have some very clear misconceptions in
physics (see Items 9 on gravity and 18 on centrifugal force,
under Statement Category #11, Physics, in Table 1).
Trend 6: Students are significantly more credulous of
UFO tales than are faculty (see Items 4 and 19,
under Statement Category #6, Extraterrestrials, in Table 1), although both groups are
equally open to the possibility of extraterrestrial intelligence (see Item 34).
Trend 7: Students and faculty are
equally well informed on some general science issues (see Items 15, 40, 59, and 63,
under Statement Category #10, General Science, in Table 1).
Trend 8: Students are significantly more credulous toward
alternative medicine than are faculty (see Items 12, 51, and 54,
under Statement Category #14, Alternative Medicine, in Table 1), although
both are equally cognizant of the link between smoking and
lung cancer (see Item 52, under Statement Category #13, Medical Awareness,
in Table 1).
Trend 9: Students are
more credulous toward occult beliefs than are
faculty (see Items 13, 22, and 45,
under Statement Category #2, Occult Beliefs, in Table 1).
Trend 10: Faculty are about as credulous toward
some folk beliefs as are students (see Items 28 and 62,
under Statement Categories #1, Folk Beliefs, in Table 1).
Note that both students and faculty show unjustified credulousness towards graphology (Item 17 in Category #2), subliminal tapes (Item 11 in Category #4, Brain and Mind), chiropractic (Item 54 in Category #14), acupuncture (Item 70 in Category #14), and the stress-disease connection (Item 21 in Category #13). In addition, both students and faculty are misinformed about fairy photographs (Item 29 in Category #1), the 1964 NYC power blackout (Item 62 in Category #1), the CIA-UFO connection (Item 41 in Category #6), and the Vitamin C content of canned vegetables (Item 32 in Category #8, Nutrition Awareness). Trends 1, 6 and 9 may be due to media promotion of the occult. As Singer and Benassi (1981) point out (p. 55), ``the most serious side effect of occult beliefs may be that they are inherently discrediting of science.'' Trends 2 and 3 should not be surprising within the framework of William Perry's theory of intellectual development (Perry, 1970), which predicts that most undergraduates are dualistic thinkers reliant on authority figures to reveal the ``truth'' to them. Trends 4, 5 and 8 may be attributable to deficiencies in science and health education. According to Singer and Benassi (1981), these deficiencies are: (a) science is often taught as a set of facts and concepts to be acquired by rote memorization; (b) basic knowledge is lacking (although there are some areas, such as evolution and continental drift, which seem to be accepted and understood, as trend 7 demonstrates); (c) science is perceived by many (perhaps because of the current controversy surrounding some issues) as being subjective and unable to assess or predict the validity of ideas; (d) analytic ability is lacking because our math and science classes frequently do not provide the opportunity or encouragement for students to develop critical thinking skills.
Comparison of Results
Since the survey carried out here was patterned after that of Morris (1981), it is useful to compare the results presented here with his. Seventeen of the statements are very similar to or identical with those used by Morris (1981) in his survey (Items 1, 2, 7, 8, 13, 14, 15, 17, 19, 24, 38, 45, 46, 52, 55, 61, and 68; see Table 2). Unfortunately, there are two problems in doing a quantitative comparison: (a) Morris published only means without standard deviations, and his raw data are no longer available (Morris, private communication, 1993); (b) Morris used an 8-category scale rather than a 6-category scale: absolutely believes (category 8), strongly believes (category 7), moderately believes (category 6), slightly believes (category 5), slightly disbelieves (category 4), moderately disbelieves (category 3), strongly disbelieves (category 2), and absolutely disbelieves (category 1). In Table 2, the authors' means and Morris's means are listed, along with the categories in which these means fall. For the purposes of qualitative comparison, the authors collapsed Morris's categories 1 and 2, assuming that these are similar to the authors' category 1, and collapsed Morris's categories 7 and 8, assuming that these are similar to the authors' category 6. For seven of the 17 questions, the authors' mean and Morris's mean are categorically in agreement. Both groups showed slight disbelief in dowsing (Item 1) and faith-healing (Item 7), slight belief in psychic predictions (Item 14), evolution (Item 15), graphology (Item 17), and clairvoyance or remote viewing (Item 38), and moderate belief in the connection between smoking and cancer (Item 52). For four of the remaining ten questions, the means are different by as much as two categories. Morris's psychology students were on the average considerably more skeptical of astrology (Item 13) and communication with the dead (Item 45) than RAC's astronomy students, yet surprisingly the reverse seems to be true concerning psychokinesis (Item 46). For the questions about life on Mars (Item 55) and ESP (Item 61), RAC's statements are clearly different from those used by Morris. Certainly, Morris's students were considerably more credulous about the possibility of life on Mars. Probably this is because Morris's students were asked if they agree with the statement that life will be found on Mars, whereas RAC's students were asked if they agree with the statement that life has been found on Mars. Clearly, the latter statement is false, whereas the former is possible. Similarly, the statement that ``some people have demonstrated powers of ESP'' is refutable, but the statement that ``ESP exists'' is non-falsifiable.
Another survey which examined pseudoscientific beliefs in college students is that of Harrold and Eve (1986, hereinafter HE). Six of the 29 statements they posed to a sample of 409 students are very similar to those used by RAC in his survey (Items 13, 14, 19, 45, 56, 59). Harrold and Eve used a 6-category scale: definitely true (category 1), probably true (category 2), probably false (category 3), definitely false (category 4), inconclusive evidence (category 5), and no opinion (category 6). They binned together Groups 1 and 2 (Believe), 3 and 4 (Disbelieve), 5 and 6 (Don't Know), and determined percentages for each of the these three groups. Comparison of HE's results with those of the authors for the six questions in common is also shown in Table 3. Expected values for the chi-squared test were calculated by using the larger Harrold sample as the ``control'' group. Surprisingly, HE's students were about as skeptical of astrology as the UH-Hilo faculty, a result quite different from other surveys of American adults (see below). Also note that HE's students were more credulous toward spiritualism and psychic predictions than RAC's students (p=0.001). As with UH-Hilo students and faculty, about 50 % of the sample accept the statement that the Earth is nearly 5 billion years old. It should be noted that no comparison of HE's data on belief in UFO spacecraft can be made because their sample percentages don't add up to 100 % (implying a typographical error). It is interesting that RAC's astronomy students, who are significantly more credulous (23 %) than UH-Hilo faculty (8 %) toward the idea that UFOs are alien spacecraft, are not nearly as credulous as larger samples of the population; a poll conducted by Jon Miller in 1990 revealed that 54 % of American adults believe that ``some of the UFOs reported are really space vehicles from other civilizations.'' (Frazier, 1992)
Comparison of the authors' survey data with 1990 Gallup poll data (Gallup and Newport, 1991) on pseudoscientific beliefs in American adults is shown in Table 3. Nine questions they posed to a sample of 1,236 adults are very similar to those used by the authors (Items 7, 13, 14, 34, 37, 45, 56, 61, 68). The Gallup survey uses a 3-category scale: believes, not sure, and disbelieves. To compare the Gallup data with the results of the UH-Hilo survey, the authors binned together categories 5 and 6 (Probably True), 3 and 4 (Unsure), 1 and 2 (Probably False), and determined percentages for each of these three groups. Expected values for the chi-squared test were calculated by using the much larger Gallup sample as the ``control'' group. One trend which stands out is that the percentages of American adults who disbelieve in these paranormal phenomena are consistently higher than the corresponding percentages in UH-Hilo astronomy students, but consistently lower than the corresponding percentages in UH-Hilo faculty (except for Item 34, about the possibility of intelligent life on other planets). In addition, the percentages of American adults who believe in these paranormal phenomena are consistently higher than the corresponding percentages in UH-Hilo faculty. Note, however, that the percentage of UH-Hilo astronomy students that believe in astrology (21 %) is comparable to the corresponding percentage in the larger Gallup survey (25 %) ; this is significantly greater (p=0.001) than the corresponding percentage (8 %) found by Harrold and Eve in their survey. Similarly, the percentages of UH-Hilo astronomy students that believe in psychokinesis (16 %) and communication with the dead (17 %) are comparable to the corresponding percentages in the larger Gallup survey (17 % and 18 % respectively). Moreover, there is no significant difference between the UH-Hilo faculty and the Gallup sample concerning belief in communication with the dead or belief in the ability of some people to make reliable predictions using psychic powers. The results also suggest that the students sampled by HE may have been more credulous of psi phenomena than either RAC's astronomy students or the larger sample of American adults; for example, 59 % of HE's students believed in psychic predictions (Item 14), while 38 % of HE's students believed in communication with the dead (Item 45). Note that 46 % of the Gallup sample believes in spiritual healing (Item 7), a significantly higher percentage than among either UH-Hilo students or faculty (p=0.001). However, the wording of the question in the Gallup survey seems to be emphasizing the individual mind-body connection more than healing by psychic or religious figures, and the mind-body health connection is a topic of current medical research. Another interesting comparison is that the UH-Hilo astronomy students seem to be more skeptical (p=0.001) of ancient astronauts (13 % believe, although 50 % are unsure) than either HE's students (22 % believe) or the larger Gallup sample (27 % believe). Could the idea of past extraterrestrial visitations be losing its credibility, as suggested by Harrold and Eve? Also shown in Table 3 is a comparison between the authors' survey data on the acceptance of evolution (Item 15) and data obtained from a 1985 survey of 1,992 American adults carried out by Jon Miller (Frazier, 1986b). Notice that there seems to be no significant difference between acceptance of evolution in U.H. Hilo faculty and in Miller's sample of adults with graduate degrees. One should also note that the percentage of UH-Hilo astronomy students who accept evolution (57 %) lies in between the corresponding percentages in Associate of Arts graduates (49 % acceptance) and in Bachelor of Arts graduates (63 % acceptance). Both of these are sensible results!
Note also from Table 3 that there are only four questions which are common to the Crowe, Gallup/Newport and Harrold/Eve surveys. It is apparent from this that the UH-Hilo faculty are noticeably more skeptical of these statements than any of the other three groups, thus justifying our assumption that the UH-Hilo faculty represent a more enlightened ``control'' group compared to UH-Hilo students. For two of these four questions (Items 14 and 45), there was no significant difference in response between the UH-Hilo faculty and the Gallup sample. However, for three of the four questions (Items 14, 45, and 56), there were statistically significant differences in response between the HE sample and both UH-Hilo faculty and students. As mentioned earlier, HE's students were more credulous of psychic phenomena, spiritualism and of ancient astronauts than were UH-Hilo students. Conversely, HE's students were highly skeptical of astrology, and the percentages in each category were surprisingly nearly identical to those for the UH-Hilo faculty!
Assessment
Although the survey discussed in this article was only preliminary, it has stimulated us toward designing a follow-up survey, in which we would take into account the results of feedback to the students, and in which the subset of physics/astronomy statements could serve as a group of ``control questions'' for students enrolled in General Astronomy. If we can make our science courses even more stimulating and appealing by encouraging critical thinking (in a way that is entertaining and informative), we may be able to make a greater effect on the intellectual development of a larger number of students. If we do not make the appropriate changes, and are not able to keep up with current perspectives in effective science teaching, the overall public confidence in science, which is still reasonably high, may be jeopardized, as noted by Frazier (1986a). Given statistics showing a deteriorating level of mathematics and science education in America, this truly would be disastrous. We must show that science really can be fun without being intimidating! If one takes the trouble to learn, one should discover that real science can be more exciting than pseudoscience!
Acknowledgements
RAC acknowledges his wife Debra for her assistance in designing the survey questionnaire. In addition, thanks should go to Roger Baldwin for useful discussions regarding chi-squared testing.
References
CROWE, R.A. & CROWE, D.A. (1994) The scientific method and exotic claims. In Understanding the cosmos, R.A. Crowe. Long Beach, NY: Whittier. Pp. A1-13.
FRAZIER, K. (Ed.) (1986a) Americans respect science, don't know much about it. Skeptical Inquirer, 10 (No. 4), 298.
FRAZIER, K. (Ed.) (1986b) Survey examines level of scientific illiteracy in U.S. Skeptical Inquirer, 11 (No. 2), 116-119.
FRAZIER, K. (Ed.) (1989) Gallup poll of beliefs: astrology up, ESP down. Skeptical Inquirer, 13 (No. 3), 244-245.
FRAZIER, K. (Ed.) (1992) UFOs as ET spacecraft? Belief strong, steady. Skeptical Inquirer, 16 (No. 4), 346.
GALLUP, G.H., Jr. & NEWPORT, F. (1991) Belief in paranormal phenomena among adult Americans. Skeptical Inquirer, 15 (No. 2), 137-146.
HARROLD, F.B. & EVE, R.A. (1986) Noah's ark and ancient astronauts: pseudoscientific beliefs about the past among a sample of college students. Skeptical Inquirer, 11 (No. 1), 61-75.
LEDERMAN, L. (1984) The value of fundamental science. Scientific American, 251 (No. 5), 40-47.
LETT, J. (1990) A field guide to critical thinking. Skeptical Inquirer, 14 (No. 2), 153-160.
MORRIS, S. (1981) Believing in ESP: effects of dehoaxing. In K. Frazier (Ed.), Paranormal borderlands of science. Buffalo, NY: Prometheus. Pp. 32-45.
PERRY, W.G., Jr. (1970) Forms of intellectual and ethical development in the college years. New York, NY: Holt, Rinehart &Winston.
SINGER, B., & BENASSI, V. (1981) Occult beliefs. American Scientist, 69, 49-55.
WESTRUM, R. (1976) Scientists as experts: observations on ``Objections to astrology.'' The Zetetic, 1, 34-46.