What’s the difference between science and pseudo-science?

Jennifer Raff —  May 17, 2013 — 13 Comments

Science is a systematic method of acquiring information. It depends on the idea that the natural world works according to certain principles, and that we can discover those principles through observation and experimentation. Science isn’t the the only way of knowing about the world, but we give it special respect because it works so well. (I talked more about that in my previous post).

Sometimes unscientific belief systems masquerade as science in order to claim the benefit of that special respect. In many cases (magic, ghosts, the Loch Ness monster), it’s fairly easy to tell them apart. But what about homeopathy? Intelligent design? Energy healing? Schools don’t do a very good job teaching students to recognize and understand good scientific research. Fortunately it’s really not that difficult, but it DOES take one or two more steps beyond just accepting what you read.

Why this matters
Do we really care that some people claim that the pyramids were built by aliens? Maybe not, but we should absolutely care when kids catch preventable diseases because their parents bought into the deception that vaccines cause autism.

Non-scientific, non-rational explanations are attractive because they appeal to the romantic, and can be fun to think about. It’s fine to believe what you want, but it’s dishonest and harmful to pretend that arguments are scientific when they’re not. I like how Quackwatch describes the importance of distinguishing between science and pseudoscience:

Pseudoscience often strikes educated, rational people as too nonsensical and preposterous to be dangerous and as a source of amusement rather than fear. Unfortunately, this is not a wise attitude. Pseudoscience can be extremely dangerous.

• Penetrating political systems, it justifies atrocities in the name of racial purity
• Penetrating the educational system, it can drive out science and sensibility;
• In the field of health, it dooms thousands to unnecessary death or suffering
• Penetrating religion, it generates fanaticism, intolerance, and holy war
• Penetrating the communications media, it can make it difficult for voters to obtain factual information on important public issues.

How to identify pseudoscience

The very first thing to do is to check the source of a claim. Where was it published? If it’s in a news article, does the article give a citation to a reputable journal? Then it’s probably reasonable to accept it (at least for your purposes…you’ll likely need more than a lay person’s education to distinguish between scientific articles after they’re published. If something is really really important to you, that’s the point at which you ask an expert).

Why a journal? Scientific journals are peer-reviewed, a process based on the simple idea that only experts are qualified to evaluate the work of other experts. Peer review is a pretty high standard, though it can’t always detect deliberate deception. For example, Andrew Wakefield, the former physician who fraudulently claimed to have
found a link between vaccines and autism actually did get his study published in the prestigious journal “The Lancet”. But the deception was subsequently uncovered, the paper was retracted, and Wakefield’s medical license revoked. Deception that makes it through peer review can often be identified when other researchers try to replicate or build upon the results of a scientist’s published work.

It’s important to know that there are some fake science journals out there, so if you’re dealing with a controversial subject you might consider looking up the journal here, or googling it to make certain that it’s legitimate.

But what about claims published in places other than academic journals? I’ve made a simple figure to illustrate the “hierarchy” of authority, to help you answer the question “Can I trust what this source is saying?”

We all know, of course, not to read the comments section of the Internet for any reason other than entertainment, and I would really caution you against trusting crowd-sourced knowledge on places like Reddit. Utterly shun the vileness that lurks in the comments below Youtube videos.

Read blogs skeptically. Be certain that all facts posted are cited to some source (and call me on this if you notice I post something unsupported).

Journalists, even some who specialize in science reporting, get things wrong much more frequently than you might think. So if a news source reports a science finding that you are really interested in, it can be worth checking the journal article they’re basing the report on. Or, alternatively, read several different news reports about the same story. See if they differ, and how.

Books are tricky, because you can’t necessarily tell which are peer reviewed and which aren’t. Like blogs, people can publish anything they want in a non-peer-reviewed book. Be very, very skeptical when someone cites a book as their source for a finding. Even
“textbooks” can be biased.

In addition to asking “where was it published?”, here are some warning signs to alert you that what you’re reading might be pseudoscience.

Be cautious if what you’re reading:

>Presents a claim that can’t be falsified.
The scientific method does not prove things right, it disproves wrong things. All conclusions are tentative, and subject to immediate revision if evidence is discovered that proves them wrong. Therefore, it’s a basic tenant of science that one must be able to possibly prove wrong any hypotheses. (Here is an example of a non-falsifiable hypothesis)

>Isn’t predictive.
A good scientific model, backed by many studies, can produce testable hypotheses. The theory of evolution, for example, makes certain predictions. The evolutionary relationships between many species was worked out on the basis of fossil evidence, long before DNA was discovered. Now that we’re able to sequence the genomes of these species, we’re finding that many of the predictions were true. In contrast, Intelligent Design (a form of creationism) is not scientific because it doesn’t make any predictions.

>Invokes the supernatural.
Science is based on the philosophy that there are natural explanations for natural phenomena. Any non-natural explanations (God, magic, “energy”, ghosts, anything that violates the laws of physics) are by definition non-scientific.

>Has “proven” conclusions, and ignores or explains away facts that are inconvenient.
If facts contrary to a hypothesis are discovered, the hypothesis must be re-evaluated/rejected. If a field starts with a belief (such as “Aliens built the pyramids”) and then looks only for evidence that supports this belief, it’s not science.

“The very powerful and the very stupid have one thing in common. They don’t alter their views to fit the facts. They alter the facts to fit the views.”*

>Uses anecdotal evidence. However interesting personal experiences might be, they don’t count for the purposes of scientific research. People are flawed, however well-intentioned. Their observations of the world can be distorted by incomplete information and their personal biases. A single person’s testimony might be useful for choosing a restaurant, but isn’t useful for scientific knowledge, which requires reproducibility. Furthermore, just collecting a bunch of anecdotes isn’t the same thing as a carefully designed study (e.g. http://en.wikipedia.org/wiki/Randomized_controlled_trial), because without controlling for variables you simply can’t determine cause-and-effect relationships. My mentor has a sign displayed prominently in his office: “The plural of ‘anecdote’ isn’t data”, and it’s a useful caution for all of us to keep in mind.

>Has various justifications for why it hasn’t been validated by the normal process of peer review. Often this will be a conspiracy theory about “mainstream” scientists trying to suppress findings that go against their own research. This is sometimes called
Galileo syndrome.

> Relies on “ancient wisdom”
We are better at science than ancient Egyptians or ancient Babylonians. We know more about how the world works this year than we did 10 years ago. Scientific discoveries are progressive, building upon a history of tested and rejected ideas and experiments. Generally, the most recent discoveries are the most useful because of this process. With pseudoscience, however, the opposite is true: the oldest findings are considered the most valuable, and “lost” knowledge is revered more than research published yesterday. I think that the majority of “ancient wisdom” fetishists underestimate ancient peoples. They DID have the technology and ingenuity to build the pyramids, say, or chart the movement of the stars. And since then we’ve simply built upon their knowledge, discarding what has proven to be false along the way.

>Argues from ignorance/personal incredulity
Just because a person doesn’t know how something works (yet) doesn’t mean that it’s unknowable, or necessary to invoke supernatural explanations. For example, quite often Intelligent Design creationists will point to some feature of an organism, such as the eye, and say that it’s too complex for evolutionary theory to explain…and therefore must have been “designed” by some creator. Unfortunately for this line of argument, it turns out that a person’s ignorance of a subject isn’t really a good basis for throwing out the scientific method. To use the exact same example, here’s Eugene Scott explaining why the irreducible complexity argument fails with regard to the evolution of the eye: http://www.dnalc.org/view/16982-The-Eye-and-Irreducible-Complexity-Creationism-Debunked.html

This is obviously not a complete list, but hopefully it’s enough to help you sort through the most egregious examples of pseudoscience. Please feel free to add your own suggestions and strategies in comments!

—————————————————–
** Doctor Who, “The Face of Evil” episode of the Fourth (and best) Doctor’s run. First transmitted 01/01/1977.

Sources and Further reading:

http://www.senseaboutscience.org/pages/peer-review.html

http://woofighters.org/2010/06/warning-signs-that-something-is-not-scientifi/

http://politicalcalculations.blogspot.com/2009/08/how-to-detect-junk-science.html

http://www.quackwatch.org/01QuackeryRelatedTopics/pseudo.html

About these ads

Jennifer Raff

Posts

Scientist, fighter, reader. In pursuit of the extraordinary.

13 responses to What’s the difference between science and pseudo-science?

  1. 

    Your blog is very nice, makes me wish I had a science career.. Science is food for my brain, You are the hottest scientist ever!

  2. 

    Ah yes, the dreaded anecdotal! The anti-data. And then there’s Audubon’s quote, which I especially like: “When the bird and the book disagree, always believe the bird.”

    Seriously, I’m with you on most of this post, but I do believe that science needs to remain aware of the the ENTIRE spectrum of scientific reality before getting too high up on the credibility horse:
    * that getting Journal Published can be political (ask Merzenich, or Taub)
    * that WHAT is studied to BE Journal published is predicated on funding, which is more in the hands of the popular press and science-illiterate politicians than *anyone* would like, and weighted to substance studies that can attract pharma-funding (in addition to the reality that the feet of substance studies best fit the slipper of the “placebo controlled, double-blind yada yada” definition of “scientifically proven”)
    * that some studies did not seem to be “replicate-able” until chronorhythms were factored into the equation equally (Foster)
    * that newer fields (like Affective Neuroscience) couldn’t get published much at all until they started their own Journal.

    THEN we have small study sample size to compare to the reports of the in-the-trenches “anecdotal” population who were excluded from the studies (N-24 in the sighted population), or not so extremely affected *in a particular manner* to be referred (hypo-active girls with ADD). Let’s see, confirmation bias, turf wars, reluctance to disagree with a mentor . . .

    I could go on, but I believe I’ve made my point: that black and white thinking from ANY paradigm is inadvisable, especially from the point of view of those whose lives (and quality of same) continue to be negatively affected until the blind men stop squabbling about the shape of the elephant.

    FYI: I back-linked this to “Science and Sensibility” on ADDandSoMuchMore. Nice post.

    ~~~~~
    Madelyn Griffith-Haynie, CMC, SCAC, MCC
    – ADD Coach Training Field founder; ADD Coaching co-founder –
    (blogs: ADDandSoMuchMore, ADDerWorld & ethosconsultancynz – dot com)
    “It takes a village to transform a world!”

  3. 

    i was hoping to see wikipedia somewhere on your scale of trustworthiness ;)

Trackbacks and Pingbacks:

  1. Are genetically modified organisms safe? | Violent metaphors - June 9, 2013

    […] This is exactly the sort of thing that alerts my skepticism. Remember that anecdotes are one of the warning signs of pseudoscience. […]

  2. Why you should vaccinate your children | Violent metaphors - June 15, 2013

    […] don’t even understand the germ theory of disease. All of their so-called evidence is either anecdotal, or […]

  3. Science and Sensibility – the illusion of proof | ADD . . . and-so-much-more - June 17, 2013

    […] What’s the difference between science and pseudo-science? (violentmetaphors.com) […]

  4. Natural News attacks TED for tightening standards | Violent metaphors - October 5, 2013

    […] scientific studies, learn how to distinguish between science and pseudoscience. I’ve written my own guide to this, but I also quite like this […]

  5. How to tell if an ancient DNA study is legitimate « Violent metaphors - February 7, 2014

    […] from bad ones is the same as distinguishing pseudoscience from legitimate science in general: ask where the results are published. Are they in a peer-reviewed journal? Or does the author present it as “science by press […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s