The null hypothesis argument continues...
The only problem is that aura aficionados will adhoc their way out of any failure.
Yes, they will. That doesn’t affect the results though. If the experiment shows that none of those who claim to see auras can actually do so, there will be some, probably many, who will refuse to believe it. If the results are positive, similarly there will be many who will refuse to believe it. The results will stand, all the same. I work with data, not belief, and I'd only be interested in a refutation--either way--if the critic could point to an actual error in procedure or calculation.
There are still people convinced the Earth is flat. There are still people who insist orbs are more than dust, who insist those rods aren’t a product of insects and shutter speed, and so on. Some people are fixed in their ideas and won’t listen, even when presented with proof. On both sides.
Romulus, you appear to be taking a rather micro view of the term "hypothesis" and appending it on to the term "null hypothesis".
The null hypothesis is a special case. Hypothesis is a wider term.
And while this would be consistent with Popper's evaluation of what constitutes a "good" hypothesis, such an approach does some discredit to "macro" hypotheses that also meet Popper's falsafiability test.
The null hypothesis and macro hypotheses are very different things. Falsifiability is a different thing again. The orbiting teapot is actually a good example here - if there's no way to prove the orbiting teapot isn't there, then there's no point investigating it. An experiment must have two possible outcomes, 'yes' or 'no'. If there is only one possible outcome, there's no purpose to the experiment. It must be possible to prove the 'no' as well as the 'yes'. Sometimes you find you can't prove either, but they must both have been possible at the start of the experiment. that's where falsifiability comes in. The hypotheses formulated at the start of an experiment are done after the test for falsifiability. If something fails the falsifiability test, no hypotheses are ever formulated because the experiment won't be started.
Great example - forget ghosts or bigfoot. How about Richard Dawkins' "The God Hypothesis"?
You have declared yourself agnostic. I’m a ‘don’t care’. There might be a god, there might not be. I see no way to test that short of seeing something that could only be considered a miracle. Also, there's no way to prove God doesn't exist so he fails the falsifiability test, I'm afraid. Sorry, haven’t read either side’s books on this.
We have an extraordinary claim (that God exists), a null hypothesis (that there is no evidence to suggest that He does)
Well, that’s not really a hypothesis. If you add one word to make it ‘there is no scientific evidence to suggest that he does’, it’s not a hypothesis at all. It’s a fact, and will continue to be a fact until some evidence arises. I don't mean belief, or the conversion of a rabid atheist to religion. I mean recordable data. There is none, and stating so constitutes a fact, not a hypothesis. The null hypothesis here is ‘There is no God’, and the alternate hypothesis is ‘There is a God (or Gods)’. It could be better worded, but the principle is there. It's not an insult to religion, it just means what it says. Science has no evidence to suggest there is a god.
Contrast this with scientific disapproval of the use of the word "theory" in the Discovery Institute's Theory of Intelligent Design.
Ah, the discovery institute. This week’s New Scientist has bad news for them. The peppered moth is back. But that’s a digression. Science does indeed disapprove of misuse of terminology. That's why we're arguing.
The Theory of Intelligent Design, on the other hand isn't even a hypothesis, because there's no way of declaring it invalid.
Agreed – there is no experimental structure, so they can’t formulate a null hypothesis. There is no defined test, so the idea can’t be declared either valid or invalid. They can call it a theory in the general sense of the term, but not in the scientific sense. A scientific theory is based on accumulated data, not belief. Is this relevant? It’s just another example of misunderstanding of a scientific term.
You have a misconception of the null hypothesis.
No, I haven’t. Been using it for many years, in its proper application. It’s what I do. That’ll be an appeal to something, no doubt. It’s experience, in fact.
The null hypothesis is the default position. There is no requirement to "prove" the null hypothesis, because this would not only be a pointless exercise, it is reversing the burden of proof.
No, it doesn’t reverse the burden of proof. That still lies with the claimant. The null hypothesis is an experimental tool. It’s a baseline, if you like. It represents the results you’d get if the data arose purely by chance. To reject the null hypothesis in favour of the alternate hypothesis (that there is some real effect) then the recorded data has to differ significantly from that predicted by the null hypothesis. If it does not, then the assumption must be that the data could have arisen purely by chance and is therefore nothing special. That means the null hypothesis (that there is no real effect) is proven, or accepted if you prefer the statistician’s terminology.
Take the aura test. Any test, doesn't matter. Your statistician can work out, for a given number of 'tries', how many you'd expect someone to get right just by guessing. That baseline formulates your null hypothesis; 'the results are no better than those obtained by guesswork'. You even have a graph already set up for the 'guess' data. Now, your aura-seer has to produce a significantly better-than-guess dataset to be taken seriously. At least to P<0.05, although for a claim this extraordinary you'd really want to see significances better than that.
In other words, we can "assume" the null hypothesis until such time that evidence becomes available that confirms the contrasting claim. We do not have to test the null hypothesis.
Nobody ‘tests’ a null hypothesis, because it is the test. It doesn’t have any use on its own. It’s applied to something you want to test. It isn’t testable on its own. It has a specific application in science, and is of no value as some stand-alone idea.
Of course, this does not mean that evidence that confirms the null hypothesis is not welcome.
The results, welcome or otherwise, must be accepted. Rejecting unwelcome information is not the way science works. I’m out for proof, remember, and it has to be watertight, especially given the nature of what I’m trying to prove. I can't afford to cut corners.
Your ghost example is a good one for that. But you went about this the wrong way.In your example, you are not testing for the null hypothesis at all. You are testing other possible explanations (hypotheses, even) by a filtering process in order to be able to say, "Looked for rodents - couldn't find evidence for them. Looked for flickering lights - couldn't find evidence for them."
No, I am not testing a null hypothesis at all. Nobody does. I’m testing a claimed haunting. The null hypothesis is a part of the test procedure, not the thing being tested.
Each of these has their own possible null hypothesis, although it's possible for some that the claim may in fact be the default position. In which case any other explanation becames the extraordinary claim.
Now you’re saying that each item within an investigation should have its own null hypothesis, which is a bit of a switch from an overall ‘The Null Hypothesis’, isn’t it? There’s no need for each item within an investigation to have its own hypothesis. Each investigation comprises a datapoint. There can be no statistical analysis of a single investigation. That can only be applied to a mass of accumulated data, over many investigations. There’s no need to set up each component of an investigation as an experiment in its own right, because I have only one thing to test – is it a haunting or not – and I then have to determine whether there is a ‘normal’ explanation for the observed effects. You’ve already said it looks to you like a ‘filtering’ process, well how much worse would it be if every component of an investigation had its own, separate experiment? To be honest, any scientist who set up an experiment that way would be a laughing stock.
Example: I'm on a country road and I see two bright lights shining on the road. The null hypothesis is that it is headlights. The extraordinary claim is that it is something other than headlights. If I don't assume the null hypothesis, there is a fairly great chance that I'll be run over.
Here, again, you are not assuming ‘The Null Hypothesis’, you are formulating specific null and alternate hypotheses in a specific situation. Whether you currently think the null or alternate hypotheses are correct is irrelevant to your safety though – if it’s coming towards you, whether it’s a thing with headlights or something else, it would be prudent to get out of the way. You can determine which of your hypotheses is correct as it passes.
I am intrigued to know where you are going with this. It started when I pointed out that you couldn’t apply some overall ‘Null Hypothesis’ idea to an overall question (Is there a Bigfoot) and you insisted I was wrong. Yet in your example, you apply a specific null hypothesis to a specific situation, which is the correct way to do it. Further, you now suggest I should break up one investigation into a subset of a dozen or more separate experiments, each with their own null hypotheses. That’s going too far the other way.
I could tell you how long I’ve been a scientist, how many degrees I have, how many years I spent researching and lecturing. I could tell you where to find definitions of the null hypothesis and other kinds of hypotheses, and how to detect (and avoid) the type 1 and type 2 errors associated with this approach. You will dismiss all that as ‘an appeal to authority’, ignore it, and continue to argue in circles.
Please, continue. I enjoy arguments.
15 comments:
Oh you found my post? Good work. Can I have it back? I hadn't actually finished typing it up.
I thought it ended somewhat abruptly. If you want to add more, I can edit that post to add it in.
Probably not. I think that it said what I needed to say.
I could tell you how long I’ve been a scientist, how many degrees I have, how many years I spent researching and lecturing. I could tell you where to find definitions of the null hypothesis and other kinds of hypotheses, and how to detect (and avoid) the type 1 and type 2 errors associated with this approach. You will dismiss all that as ‘an appeal to authority’, ignore it, and continue to argue in circles.
You didn't really write that, did you? Please tell me that you're not this pompous?
Why is it pompous to declare my credentials? Haven't you referred to your history and philosophy of science studies, as evidence you know what you're talking about?
Why is that different?
We're arguing over a scientific term, yet it's pompous for me to say I've been a scientist, using those terms, for many years?
Please clarify.
I mentioned that I'd studied some HPS. That was relevant in the context that I used it.
At no stage was I questioning your credentials. So what motivated you to produce them?
Many scientists I know, in several discplines, would have reacted differently to your approach.
If I had been one of them (and I don't regard any of them as pompous), then dictating how research should be carried out would have earned you a very short answer. Star-ratings on posts would have finished any conversation with those scientists before it started, for example.
I choose to hear what everyone has to say, because when my approach is questioned I can learn from it. Particularly, I can work out what will eventually satisfy a skeptic, and what kind of evidence I need to concentrate on collecting. Often, the answer is 'none', but not always.
Now, we are not arguing about the paranormal here. When we were, my credentials were irrelevant. A PhD in 'woo-ology' isn't going to affect those arguments one way or the other.
We are arguing about the application of a specific scientific technique. One I use all the time. Should I not state that? Should I not specify my qualification to discuss the subject? Is it out of context to do so?
I don't think it is. I would not consider investment advice from someone with no background in investments. Would you?
Why, then, in a discussion on scientific technique, is it out of context to state that I do have the necessary background?
Would you prefer I said 'I know what I'm talking about, so there'?
That would make it far too easy.
How far would you like to argue this? We're a long way from anything paranormal here. We're not even arguing science any more.
I'm not sure what your point is. I expect you to make your points, which you do very well, I think. Your credentials, while interesting and admirable, are not relevant because I never questioned them.
Let me put this another way.
If I had mentioned that I dropped out of university after one year, commenced an apprenticeship in floristry followed by six years as a florist after which I moved into a management position with a fast food chain for 6 more finishing with my current role as an entry level administrator with an investment bank, would we be actually having this conversation, or would you have long ceased this dialogue?
Or, another example. If an ex-stage magician criticises a scientist for flaws in a study, would you immediately dismiss such criticism as worthless?
I suspect that I know your answers to these.
If I had been one of them (and I don't regard any of them as pompous), then dictating how research should be carried out would have earned you a very short answer.
Romulus, you overlooked Jacques Benveniste's titanically flawed research methods to focus on a stunt by James Randi and his background as an ex-stage magician. What am I supposed to think?
Star-ratings on posts would have finished any conversation with those scientists before it started, for example.
Humourless gits, I say.
I don't think it is. I would not consider investment advice from someone with no background in investments. Would you?
Would you accept movie reviews from reviewers who have never been film directors? A book on General Franco by someone who has never been a fascist dictator?
"Investment advice", incidentally is a tightly defined term in law over here, and may only be provided by professionals with a minimum level of education. So it wouldn't be up to me to choose.
If I had mentioned that I dropped out of university after one year, commenced an apprenticeship in floristry followed by six years as a florist after which I moved into a management position with a fast food chain for 6 more finishing with my current role as an entry level administrator with an investment bank, would we be actually having this conversation, or would you have long ceased this dialogue?
Actually, I knew you were in investment banking from the outset. It's on your profile. When discussing the paranormal, it's not relevant. Neither are my credentials. When you tell me I'm wrong about details of scientific protocols, both our credentials on the subject are relevant. Not just to us, but to anyone else who might happen across this conversation.
Anyone can discuss the paranormal. It doesn't take qualifications to do that. It does take qualifications to set up a scientific study. Not my rules.
Or, another example. If an ex-stage magician criticises a scientist for flaws in a study, would you immediately dismiss such criticism as worthless?
Ah, but that's not all he does. You've read his website. How often does he allow that he might be wrong about his conclusions? Is that scientific?
I don't know all the tricks of the stage magician, so if a magician says 'I can replicate that without paranormal powers', I want to see it.
I also want to see how it's done so I can then watch for the same trick in someone who claims to have paranormal powers. Naturally, magicians aren't keen to give away their secrets, especially since I might then say, in an experiment, 'That's not paranormal. He did it by...' and proceed to give details.
You might say 'Well why not have a magician in on the experiment?'
No use. The magician might cry 'fake' but I can't then check if the subject really was faking unless the magician reveals how he does it. The magician might produce the same visible effect by sleight of hand, but that does not prove that the subject did it the same way unless I can observe him doing it. What do I report? 'He faked it, but I can't tell you how'. Wouldn't the guy's supporters just have a field day with that? I'd be no further forward, in fact I'd be better off not doing that experiment at all.
No, I would not let the magician advise on the design of the experiment. That's what statisticians are for.
Would you accept movie reviews from reviewers who have never been film directors? A book on General Franco by someone who has never been a fascist dictator?
Yes, because neither of those things matter very much. If a film review is a load of crap I can ignore it and move on. If the book on General Franco was by an historian, I'd likely take it seriously. But it doesn't matter to me if it's wrong.
Investment advice, and the way I do my job, matter a lot to me. (investments are one of my main income sources).
"Investment advice", incidentally is a tightly defined term in law over here, and may only be provided by professionals with a minimum level of education.
Investment advice isn't so tightly defined over here, I think. Although I don't seek advice from investment managers anyway, so I can't be certain of that. Nothing against investment managers, it's just that I'm doing as well as I need to on my own.
Humourless gits, I say.
Some of them are, yes. Especially where work is concerned. Probably for the best: one of them works with deadly bacteria. I wouldn't want a practical joker in his job.
I don't suppose that we'll get any agreement on this. I don't really hold much sway with scientific credentials when there are scientists like Gary Schwartz who received his PhD from Harvard and is a professor of the psychology department at the Uni of Arizona running around.
Schwartz, incidentally, would probably be able to put together a study confirming the existence of ghosts for you. But I daresay, you would be almightily pissed off with the methods that he'd use. He's already stretched the bounds of human credibility by supposedly validating the "gifts" of John Edward and Allison DuBois in extremely shoddy studies.
And Randi (non-scientist that he is), much as you dislike him, well, we already know that he has a cheersquad of all the right scientists ready to support his knowledge of the scientific method.
Anyway, I probably overreacted with the "pompous" call. But if what you wanted to do was draw my attention to your track record, you could have just said this:
I could tell you how long I’ve been a scientist...
and left it at that. I didn't really see any need for what was, quite frankly, overkill. If I was an amateur psychology type (which I am NOT), I would probably have formed a few more conclusions about you which I'm sure are not correct.
Actually, I knew you were in investment banking from the outset. It's on your profile.
I know. You occasionally view my blog through my Blogger profile, which flatters me a little.
However, I have to be partially honest and say that in that hypothetical spiel about myself, only two things there were true and you know what one of them was. See if you can spot the other one.
I have read some of Gary Schwartz's studies. I don't think he's deliberately biased his tests but also I think he might have trusted his subjects more than I would have. Those stage psychics are big names, and have big TV companies with big lawyers behind them. That's going to make anyone nervous. So maybe he let them off with a few transgressions he shouldn't have. Enough to invalidate his tests, even.
Besides, if John Edward refused to take part in a test because he wasn't allowed to 'make adjustments', then would it really hurt his TV ratings? I doubt it. His fans already want to believe, and already think science shouldn't meddle with this sort of thing.
But then, unleashing forces we can't control and tampering with the fabric of life is the fun part of science. The day-to-day part is somewhat dull.
That's the best part of going it alone. I have no boss. I can't be fired. If someone gets huffy when their ghost turns out to be a worn motor in the fridge, then they can huff away to their heart's content. Actually, most people are relieved. It's the ones who planned to cash in on the ghosthunters who get upset. No sympathy from me. If they want to complain to my superiors, well that's the Inland Revenue, and they don't give a hoot what I do as long as they get their taxes on time.
Still, I'd be wary of taking on someone with big lawyers backing them up. Even if they were proved fake, I'll bet those lawyers could tie it all in knots for years, and me with it.
Anyway, I'm digressing.
Well, the one thing you definitely are, is in investment banking. The other one? You once mentioned you'd done a year of HPS, so I'd say you dropped out of university. The florist and the fast food would be the lies, in that case.
Don't be too flattered - I call in to your blog through your profile because I'm too lazy to remember the URL. I get there by clicking on your name in the comments list.
Very good, Romulus. But this is the new anonymous age, so I won't confirm or deny, but you are perceptive.
I'm flattered when anyone reads my blog. My readership figures are unfeasibly low.
Word verification for this comment: imfwoo
I'd think this was scary if I attached significance to coincidences, which I don't.
One of the tricks a stage psychic uses: a comment you made in passing, some time back, and forgotten, can come back as if it's news. Nothing paranormal, just a good memory.
I said this:
You once mentioned you'd done a year of HPS, so I'd say you dropped out of university
A stage psychic might have said:
I have the feeling of something incomplete... (and so on).
If I'd thought of it, I could have used it as a demo. Ah, a missed opportunity.
I think all stage and TV psychics, indeed any of the corner-shop psychics and any who do it for money, are fake. I think this because they never fail. If they really did rely on particular spirits turning up, then surely, once in a while, there'd be a show where no spirit related to anyone in the audience would turn up. Simple arithmetical averaging would demand that it happen, now and then. It never, never does. Worse, they always find exactly the right number of spirits to fill a half-hour show. Do they book in advance? In a sense, yes. they 'book' the information from the participants before the show. Then fill in with a little response-reading.
The corner-shop ones have a load of tricks in addition to the stage psychic's tricks. Any telephone or internet reading can't possibly work, not by any reasonably postulated mechanism, normal or paranormal. I don't conclude from that that all psychics are fakes, because that's like saying 'I've seen a hundred white swans, so all swans are white.' I can't conclude that unless every psychic, everywhere, is tested. But I do conclude that if they're infallible, then statistically they must be fake. Still, the tricks are easy to spot once you know what they are.
There's a really good video on YouTube where Penn and Teller expose a fake psychic (Yes, they're magicians, but they're not interfering in anyone's scientific work. It's their investigation, they set it up and they set it up very well indeed).
Proving a psychic is real would take a lot of strong, repeatable evidence. Proving a fake is easy. It doesn't even need any kind of experimental setup, just an open mind, open eyes and sharp hearing. One video report will do it. It also helps if you don't care who you offend in the process.
Tht does seem biased against the psychics, but that's not unfair. The 'claim' lies with the psychic, not the investigator. Trouble is, a real psychic wouldn't be a materialist. They won't care about proving themselves. They won't be on the TV, in a corner shop, or at the end of a phone line.
So we might never get to test anyone but the fakes. Still, if it gets a few frauds off the game, it's a good thing.
Cold reading. I love it. I wonder if there's a course you can do somewhere? It would impress the hell out of chicks at parties.
No need for courses. Just chat beforehand, give them ten minutes to chat to someone else on another topic, and they've forgotten everything they told you.
Then take visual cues for the details. A little practice is enough for party use. They'll overlook your errors as soon as you hit on the right answer.
If they can tell you a load of detailed stuff without that 'primer' chat - on first meeting - let me know! Especially if they can give full names and descriptions.
Unless the party's in your house, the subject has been chatting to other family members and you have family photos on display. Then my interest might wane somewhat.
If they mention my name, send me their phone number. It's not a name that often crops up as a guess.
Post a Comment