Randi’s Prize: Answering Chapter 4

Chapter Four:  Uncertain Science

This chapter deals with parapsychological experiments in general, rather than mediumship specifically, as the previous chapter. We are run quickly past a number of claims and rebuttals without dealing with any in detail.

Repeatability

Of the problems the most hard-hitting is probably the fact that experiments are not repeatable but unfortunately this huge problem does not get discussed but rather ignored. Perhaps McLuhan simply chose to believe assertions to the contrary?

He mentions the card-guessing experiments of JB Rhine, conducted in the 1930s, and tells us of Hubert Pearce, a theology student, who could consistently and repeatably demonstrate his ESP by scoring, on average, 33% hits where 20% was expected.

He also tells us of the Ganzfeld experiments, conducted in the 1980s onward, where, on average, people score 33% instead of 25%. The Ganzfeld is a method of creating a state of mild sensory deprivation which is supposed to enhance someone’s ability to receive extra-sensory information and thus to enable better scoring.

Curious, isn’t it? Decades pass during which parapsychologists develop a method to increase scoring but… The increase is worse than what was achieved at the time by using a “star subject”.

McLuhan tells us that card-guessing was abandoned because it was to boring, just sitting there calling out one guess after the other. In the Ganzfeld experiments, someone has to endure 20 minutes of sensory deprivation for a single guess. I am not sure how that relieves the problem.

I wonder if it may be one thing that distinguishes skeptics and believers, that skeptics have a higher need for internal consistency?

Bad statistics are a serious problem in parapsychology as they can create the impression of an effect where there is none. Naturally, not all criticisms are correct. McLuhan incorrectly generalizes rebuttals of some criticisms to mean that such criticisms as a whole are unwarranted. Looking at recent works like Bem’s Feeling the Future, it is obvious how misleading that is.

One thing that stood out to me is how McLuhan speaks with 2 voices. He generally makes an effort (or a show?) of considering both sides. Sometimes he even intimates that these argument affected him. Yet every so often a different attitude breaks through. Then he tells us why these arguments are made. Not because they are true or reasonable but only to create doubt.

In-Depth Controversy

The first controversy that is addressed in-depth is the “sense of being stared at”. Unfortunately this is not one I have studied and so I will not comment on it. I intend to do so at some time but not in the next few weeks.

The next controversy concerns Sheldrake’s psychic dogs. This has already been examined on this blog.
Someone who goes to the original articles and actually evaluates the data for himself should be able to see past Sheldrake’s wall of make-belief but McLuhan completely falls for his spin and retells it as such.

He is so faithful to that version that he even follow Sheldrake in making nasty attacks on a skeptic who had the bad judgement of taking the claims seriously enough to conduct his own investigation.

After this low of investigative effort comes a more extensive exploration of the Ganzfeld experiments. These are in many ways amongst the best parapsychology has to offer. Many other results shrivel to nothingness under scrutiny or are simply unrepeatable which means we have to take them on faith.

By comparison, this series of experiments is a shining example of methodological rigor and solidity. Some time I will make a post on why I don’t believe that there is no real effect there. I expect they will eventually end up like Rhine’s experiments in the 1930s. Never fully explained but simply abandoned.

McLuhan quotes the same skeptic praising these experiments who he had just a few pages earlier accused of trying to sabotage Sheldrake’s research. He seems completely oblivious of the inherent contradiction.

Finally, there comes the remote viewing experiments called the Stargate project, performed by the US government. Here things get more mixed. Eventually this was a debate between Ray Hyman(skeptic) and Jessica Utts(believer). McLuhan, of course, finds the believer convincing, never realizing the gaping holes in her arguments.

Feeling the Future: Part 2

In my first post on Feeling the Future, I discussed mainly how it’s misuse of statistics related to science in general. I said little about how exactly the statistics were misused. My thinking was that a detailed examination would be too boring for the average reader.
I still think that but nevertheless I will spread out exactly how we know that Bem misused statistics.

The Problem Explained

The good news is that you don’t need to know statistics to understand this problem. You surely know games that use dice. Something like monopoly for example. In that game you throw 2 dice that tell you how far to move.
What if someone isn’t happy with the outcome and decides to roll again? That’s cheating!

Even small kids intuitively understand that this is an advantage, something that skews the outcome in a direction of one’s choosing. There’s no knowledge of statistics or probability theory necessary. While the outcome of each roll is still random,there’s a non-random choice involved.

If you roll 3 dice and pick 2. That’s the same thing, right?

How about we roll 4 and then pick 2 but with the stipulation that the 2 remaining dice must be used on the next? Again there’s a choice involved. Within limitations the player can choose
how to move which allows an advantage. The player’s moves are not longer random.
This is despite the fact that the dice rolls are random and none are discarded.

Now we’re ready to get to Feeling the Future.
The results presented were very unlikely to have arisen by chance. Therefore, the argument goes, they probably didn’t arise by chance. Which means there must have been some unknown factor influencing the outcome.

You may realize that this is a shaky argument. Just because something is unlikely does not mean it doesn’t happen. The impossible doesn’t happen but the unlikely, by definition, must and does. The unlikely is set apart from the likely merely by happening less often.
Then again, the impossible is only impossible as far as we know. And that we’re wrong on something is at best unlikely, if that. In reality, as opposed to in mathematics, we’re always dealing with probability judgements, never with absolutes.
In other words, that argument is all we have. It is used in the same way in almost every scientific experiment.

So the argument is solid enough. In fact, I believe that there is something other than chance involved. Of course, dear reader, if you didn’t know that already you must have skipped the beginning of the post.

Bem’s experiments each had, according to Feeling the Future, 100-200 participants. In reality, at least some of them were assembled from smaller blocks of maybe around 50. This is a problem for exactly the same reason as the dice examples. Even if the outcomes in every block were completely random, once hand-picked blocks are assembled to a larger whole, this whole no longer is.

Proof that it happened

How do we know that this happened? This doesn’t require knowledge of statistics either, just a bit of sleuthing.
First we note what it says in footnote about experiment 5

This experiment was our first psi study and served as a pilot for the basic procedures adopted in all the other studies reported in this article. When it was conducted, we had not yet introduced the hardware based random number generator or the stimulus seeking scale. Preliminary results were reported at the 2003 convention of the Parapsychological Convention in Vancouver, Canada (Bem, 2003); subsequent results and analyses have revised some of the conclusions presented there.

Fortunately, this presentation is also available in written form. Unfortunately it is immediately obvious that it doesn’t present anything corresponding to experiment 5.
The presentation from 2003 reported not 1 but 8 experiments, each with at most 60 participants. The experimental design, however, matches that reported in 2011.
The 8 experiments are grouped into 3 experimental series, so perhaps he pooled these together for the later paper? But no, that doesn’t work either.

I could write several more paragraphs of this kind, trying to write up a logic puzzle full of numbers as if it were a car chase. But my sense of compassion wins out. I know I would merely bore you half blind, my dear readers, and I won’t have that on my conscience.

Therefore I shall only give my answers as one does with puzzles. Check them with the links at the bottom if you like. I could easily have overlooked something or made a typo.

Experimental series 300 of the presentation is the “small retroactive habituation experiment that used supraliminal rather than subliminal exposures” that is mentioned in the File-Drawer section of “Feeling the Future”.
Experiment 102 with 60 participants must have been excluded because it has 60 rather than 48 trials per session.
Experiments 103, 201, 202, 203 combined form experiment 6. They have the same number of participants (n=150). Moreover, the method matches precisely. 100 of these 150 were tested for “erotic reactivity“. This is true for experiment 6 as well as the combination.
Experiment 101 could be part of experiment 5 but there aren’t enough participants. Additional data must have been collected later.
Note that the footnote points to “subsequent results“.

Warning signs

Even without following up the footnotes and references there are some warnings signs in Feeling the Future that hint that something is amiss. For example.

The number of exposures varied, assuming the values of 4, 6, 8, or 10 across this experiment and its replication.

The only reason one would change something in an experiment is to determine if this one factor has any influence on the results. Here we learn that a factor was varied but there is neither reason nor justification given. Much less results.

These two items were administered to 100 of the 150 participants in this replication prior to the relaxation period and experimental trials.

The same thing applies here. A good experiment is completely preplanned and rigidly carried through. There’s no problem with doing less formal, exploratory work to find good candidate ideas that merit the effort necessary for a rigid test. But such exploratory experiments have almost no evidential weight.

Such warning signs are also present in the other experiments described in Feeling the Future. That could indicate that the same thing was done there as well. But don’t make the mistake of assuming that this issue is the only one that invalidates Bem’s conclusions. There’s also the issue of data dredging which is like deciding which card game to play depending on what hand you were dealt. Small wonder then, if you find your cards to be unusually good, according to the rules of the game you chose.

In terms of an experiment that means analyzing the results in various ways and then reporting those results that favor the desired conclusion. That Bem did this is also evident from a comparison of the 2003 and 2011 description of what is apparently and purportedly the same data.

Particularly worrying is that Bem has explicitly and repeatedly denied using such misleading methods. I shall restrain myself from speculating about what made him deny such an obvious, documented fact. It does not have to be dishonesty but none of the other possibilities is flattering, either.

There’s a common conceit among believers that skeptics don’t look at the data. Whenever someone claims this, ask them if there is anything wrong with Feeling the Future and you will know the truth of that.

Sources:
Bem, D. J. (2003, August). Precognitive habituation: Replicable evidence for a process of anomalous cognition. Paper presented at the 46th Annual Convention of the Parapsychological Association, Vancouver, Canada.
Bem, D. J. (2011). Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology.

Modern Mediumship Research: Summary

In conclusion to this series I probably should say a few words on what I have learned.

For one, despite strong claims, the modern mediumship research (of about the last 15 years) offers no support for the reality of anything except well-known psychological effects.

But I am unsure to what a degree this constitutes evidence against mediumship. After all, maybe the researcher failed to find positive evidence because they are incompetent and not because it doesn’t exist. But that is a rather hefty charge.
Another possibility is that you just don’t find real mediums anymore.  Maybe the lack of critical thinking among contemporary believers has allowed frauds and fools to crowd out the real talents. That’s not a very flattering possibility and will probably not go over well with believers..

But is there any nice way to explain where the evidence is off to? In principle mediumship should be dead easy to demonstrate (no pun intended).

Supposedly there were clinching experiments conducted at the beginning of the 20th century, 100 years ago which I have never reviewed. Were they really that impressive?
It seems more parsimonious that they were more along the lines of Robertson and Roy. With the occasional, inevitable fraud or perhaps fluke strewn in.

These researcher back then certainly couldn’t convince their contemporaries, perhaps for the same reason that modern mediumship researcher can’t? The charge of dogmatism against a generation of scientists that saw the rise of quantum mechanics and general relativity seems strange.

But historical claims are not what this post is about. It’s not even about scientific experiments or theories that directly challenge the underlying assumption of mediumship. It’s not about how mediumship can be regarded as a falsified idea.

Suffice it to say that the alleged modern evidence for mediumship is not. It doesn’t even suggest the existence of anything unexplained.

My Personal Opinion

Personally, I find the data whether coming from skeptics, believers or simply the non-involved to be convincing as to the non-existence of mediumship. It’s not just the more general, theoretical deductions from what science, especially biology and neurobiology has learned in the last 200 years. If dedicated believers these days can’t find a medium, if promises of money don’t lure out a real medium, what could there be?
And even if I’m wrong and there is be something there, where I haven’t looked, in the historical reports, in the absence of a genuine medium volunteering to aid in research there is nothing to be done. We can only wait until one comes forward or neuroscience finally stumbles across… something. Anything!

If you happen to have a genuine medium on your hand, why not do a test? You only need a bit of time and the willingness to put in a little effort. Here’s a primer on how to design a solid test that doesn’t end up replicating some known psychological effect.

 

Tsakiris experiments

There’s one final thing I want to mention. It goes here, because it didn’t fit anywhere else. It’s about an experiment by pod-caster Alex Tsakiris. He put together a little mediumship experiment around the end of 2008 / start of 2009.
The experiments were not that well designed but I won’t talk about that as the full details always remained sketchy. Information only dribbled out on his pod-cast and in his forum.

Two trial runs had been conducted and it had been announced that after the third was in progress. Results had been positive but the controls were lose. Controls meaning devices to rule out alternative explanations, mainly ways of cheating. The controls were persistently tightened and then something happened. It was never revealed what exactly, only that it was somehow surprising. Was it merely negative results or was some even caught hoaxing?
The fact remains that trial three was officially never finished for an unknown reason. Tsakiris announced that he had by no means aborted the experiment and intended to continue after doing some revamping. No results, no admission of failure, criticism diverted with promises for the future. And that was the end of that. That was the last experiment he ever reported.

Tsakiris has also tried himself at another paranormal phenomenon, psychic dogs, with an equal lack of success. This seems indeed to have changed his outlook somewhat. He now maintains that it is not about the data. See, that’s the problem with skeptics. They don’t care about the data. For them it’s all a matter of worldview.
The paranormal has already been proven therefore it’s completely pointless to run experiments.

Seeing this development play out on the pod-cast and the forum was eye-opening for me. The pattern is recognizable in with many parapsychological researchers but here I could see it unfold. It has certainly given me a new appreciation for the human capacity for delusion.

Modern Mediumship Research: Negative

In this post I am going to deal with results that speak against the reality of mediumship. We begin with mediumship tests where a negative outcome was admitted.
For starters there’s a study by Jensen and Cardena. It involved only one medium, however. So there’s not really much we can conclude about mediumship in general.

A larger study was published by O’Keefe and Wiseman. It involved five mediums and 5 sitters (a jargon term for client). It discusses methodological issues in great detail and can be downloaded from Wiseman’s website. It is recommended reading for anyone thinking about putting together their own medium test. In fact, similar methods could be used to test someone who claims telepathy or just merely being a good judge of character.

One argument against such negative studies is that they simply didn’t have a genuine medium on their hands. This, however, is not a very common argument in my experience. Perhaps because it is uncomfortably close to accusing the tested mediums of fraud, or perhaps because it leaves the implication that there is

The other more common argument is that the conditions didn’t permit mediumship. Usually, some factor is said to have blocked the ability, such as the presence of skeptics. This has by now turned into a muchly ridiculed cliché.
Others say it was simply that the unfamiliar surroundings of the lab and strict limitations of the protocol made the mediums uncomfortable. This has been compared (by males, of course) to the inability of people to have sex under such conditions. I find it amazing that in this day and age there are still men who are unaware of the existence of porn.
As a remedy, Julie Beischel of the Windbridge Institute argued that O’Keefe and Wiseman should have tested their mediums before testing them to ensure that they can pass the test. It sounds better if you call the pretest a “screening”.
Some more cerebral believers argue that the conditions for paranormal just so happen to be the same as for error or fraud (though they will, of course, not put it so bluntly). Such an argument is implicit in Kelly and Arcangel’s paper when they talk about “priming the pump”.

Challenges

Perhaps one might include skeptic challenges as another negative piece of evidence. These challenges are meant to call out people who claim abilities they don’t have. The most famous of these, and the blue-print for most others, is James Randi’s Million Dollar Challenge.
The rules are both simple and scrupulously fair. Whoever wishes to take up the challenge needs not only claim some ability but design himself a task, impossible to ordinary people. So rather than just claim mediumship, one would need to say, for example, ‘I can tell if someone’s parents are alive or dead.’ Of course, the conditions would have to be so as to preclude ordinary means of inference, however unlikely. And it would have to be stipulated how many tries there would be and how many would have to be successful.
Basically, one has to convince James Randi and his advisors that there is no way to do the promised feat with ordinary means. Once they accept this, the claimant has a legal contract that stipulates payout on success.

So far, the money has been quite safe.

The arguments against the challenge are numerous but ill-informed. Some think that whether or not someone wins is a judgement call by James Randi. In fact, the rules explicitly forbid any judgement calls. Success or failure must be obvious to anyone.
Others accuse Randi of dishonesty. They say he only allows weak claimants or even that he manipulates the result. This is some rather outrageous slander but even so, there are other challenges. In any case, one can also contact parapsychologists. Yet by all appearances, they don’t have any potential winners either
Naively, one would expect something like Randi’s challenge to be treated as the X-Prize of parapsychology. But what if there are no humans with super-powers? Then those who claim them would be frauds or lunatics. The reaction to the offer of a million dollars certainly does nothing to dissuade one from considering that possibility.

Still, it is true that just because no one has risen to the challenge, this does not disprove mediumship. One might claim, for example, that a true medumistic gift always comes with extreme shyness.

Science?

Finally, there is also the fact that what science has revealed over the last centuries about the world, about us and about our place in the world is rather inconsistent with mediumship. Not so long ago science believed that there was a life-force that distinguished living from non-living matter. Today this is considered refuted.
Something similar is true for the human mind, which today is considered to be the workings of the brain.

Not only has science not uncovered support for mediumship, it has refuted some of its core tenets.

In regards to this some will point out that science has been wrong before. This is quite true and, indeed, science is nothing if not a quest to be ever less wrong. But here we have a case where this quest simply leads away from the proffered answer.
There are even many examples where scientists held onto ideas that should have been recognized as discredited by the evidence. Who is the dogmatic hold-out in this story?

Still, the fact remains that while science as a whole offers strong arguments against the possibility of mediumship, it cannot be ruled out. Science could be wrong and that is a simple, unalterable fact.
If the evidence indicated that it was real then one would have to find a way to reconcile it with other seemingly contradictory evidence.
However, I have never so far come across a serious attempt at this. There are plenty of demands that mainstream science should take parapsychology seriously. Demands that parapsychology take mainstream science seriously are usually met with the charge of close-mindedness.

The Universal Negative

Eventually, what all this boils down to is quite simply that you can’t prove a universal negative. You can’t test everyone and even when you test someone it only shows that they failed in this specific test on this one occasion.