The null hypothesis argument continues...
The only problem is that aura aficionados will adhoc their way out of any failure.
Yes, they will. That doesn’t affect the results though. If the experiment shows that none of those who claim to see auras can actually do so, there will be some, probably many, who will refuse to believe it. If the results are positive, similarly there will be many who will refuse to believe it. The results will stand, all the same. I work with data, not belief, and I'd only be interested in a refutation--either way--if the critic could point to an actual error in procedure or calculation.
There are still people convinced the Earth is flat. There are still people who insist orbs are more than dust, who insist those rods aren’t a product of insects and shutter speed, and so on. Some people are fixed in their ideas and won’t listen, even when presented with proof. On both sides.
Romulus, you appear to be taking a rather micro view of the term "hypothesis" and appending it on to the term "null hypothesis".
The null hypothesis is a special case. Hypothesis is a wider term.
And while this would be consistent with Popper's evaluation of what constitutes a "good" hypothesis, such an approach does some discredit to "macro" hypotheses that also meet Popper's falsafiability test.
The null hypothesis and macro hypotheses are very different things. Falsifiability is a different thing again. The orbiting teapot is actually a good example here - if there's no way to prove the orbiting teapot isn't there, then there's no point investigating it. An experiment must have two possible outcomes, 'yes' or 'no'. If there is only one possible outcome, there's no purpose to the experiment. It must be possible to prove the 'no' as well as the 'yes'. Sometimes you find you can't prove either, but they must both have been possible at the start of the experiment. that's where falsifiability comes in. The hypotheses formulated at the start of an experiment are done after the test for falsifiability. If something fails the falsifiability test, no hypotheses are ever formulated because the experiment won't be started.
Great example - forget ghosts or bigfoot. How about Richard Dawkins' "The God Hypothesis"?
You have declared yourself agnostic. I’m a ‘don’t care’. There might be a god, there might not be. I see no way to test that short of seeing something that could only be considered a miracle. Also, there's no way to prove God doesn't exist so he fails the falsifiability test, I'm afraid. Sorry, haven’t read either side’s books on this.
We have an extraordinary claim (that God exists), a null hypothesis (that there is no evidence to suggest that He does)
Well, that’s not really a hypothesis. If you add one word to make it ‘there is no scientific evidence to suggest that he does’, it’s not a hypothesis at all. It’s a fact, and will continue to be a fact until some evidence arises. I don't mean belief, or the conversion of a rabid atheist to religion. I mean recordable data. There is none, and stating so constitutes a fact, not a hypothesis. The null hypothesis here is ‘There is no God’, and the alternate hypothesis is ‘There is a God (or Gods)’. It could be better worded, but the principle is there. It's not an insult to religion, it just means what it says. Science has no evidence to suggest there is a god.
Contrast this with scientific disapproval of the use of the word "theory" in the Discovery Institute's Theory of Intelligent Design.
Ah, the discovery institute. This week’s New Scientist has bad news for them. The peppered moth is back. But that’s a digression. Science does indeed disapprove of misuse of terminology. That's why we're arguing.
The Theory of Intelligent Design, on the other hand isn't even a hypothesis, because there's no way of declaring it invalid.
Agreed – there is no experimental structure, so they can’t formulate a null hypothesis. There is no defined test, so the idea can’t be declared either valid or invalid. They can call it a theory in the general sense of the term, but not in the scientific sense. A scientific theory is based on accumulated data, not belief. Is this relevant? It’s just another example of misunderstanding of a scientific term.
You have a misconception of the null hypothesis.
No, I haven’t. Been using it for many years, in its proper application. It’s what I do. That’ll be an appeal to something, no doubt. It’s experience, in fact.
The null hypothesis is the default position. There is no requirement to "prove" the null hypothesis, because this would not only be a pointless exercise, it is reversing the burden of proof.
No, it doesn’t reverse the burden of proof. That still lies with the claimant. The null hypothesis is an experimental tool. It’s a baseline, if you like. It represents the results you’d get if the data arose purely by chance. To reject the null hypothesis in favour of the alternate hypothesis (that there is some real effect) then the recorded data has to differ significantly from that predicted by the null hypothesis. If it does not, then the assumption must be that the data could have arisen purely by chance and is therefore nothing special. That means the null hypothesis (that there is no real effect) is proven, or accepted if you prefer the statistician’s terminology.
Take the aura test. Any test, doesn't matter. Your statistician can work out, for a given number of 'tries', how many you'd expect someone to get right just by guessing. That baseline formulates your null hypothesis; 'the results are no better than those obtained by guesswork'. You even have a graph already set up for the 'guess' data. Now, your aura-seer has to produce a significantly better-than-guess dataset to be taken seriously. At least to P<0.05, although for a claim this extraordinary you'd really want to see significances better than that.
In other words, we can "assume" the null hypothesis until such time that evidence becomes available that confirms the contrasting claim. We do not have to test the null hypothesis.
Nobody ‘tests’ a null hypothesis, because it is the test. It doesn’t have any use on its own. It’s applied to something you want to test. It isn’t testable on its own. It has a specific application in science, and is of no value as some stand-alone idea.
Of course, this does not mean that evidence that confirms the null hypothesis is not welcome.
The results, welcome or otherwise, must be accepted. Rejecting unwelcome information is not the way science works. I’m out for proof, remember, and it has to be watertight, especially given the nature of what I’m trying to prove. I can't afford to cut corners.
Your ghost example is a good one for that. But you went about this the wrong way.In your example, you are not testing for the null hypothesis at all. You are testing other possible explanations (hypotheses, even) by a filtering process in order to be able to say, "Looked for rodents - couldn't find evidence for them. Looked for flickering lights - couldn't find evidence for them."
No, I am not testing a null hypothesis at all. Nobody does. I’m testing a claimed haunting. The null hypothesis is a part of the test procedure, not the thing being tested.
Each of these has their own possible null hypothesis, although it's possible for some that the claim may in fact be the default position. In which case any other explanation becames the extraordinary claim.
Now you’re saying that each item within an investigation should have its own null hypothesis, which is a bit of a switch from an overall ‘The Null Hypothesis’, isn’t it? There’s no need for each item within an investigation to have its own hypothesis. Each investigation comprises a datapoint. There can be no statistical analysis of a single investigation. That can only be applied to a mass of accumulated data, over many investigations. There’s no need to set up each component of an investigation as an experiment in its own right, because I have only one thing to test – is it a haunting or not – and I then have to determine whether there is a ‘normal’ explanation for the observed effects. You’ve already said it looks to you like a ‘filtering’ process, well how much worse would it be if every component of an investigation had its own, separate experiment? To be honest, any scientist who set up an experiment that way would be a laughing stock.
Example: I'm on a country road and I see two bright lights shining on the road. The null hypothesis is that it is headlights. The extraordinary claim is that it is something other than headlights. If I don't assume the null hypothesis, there is a fairly great chance that I'll be run over.
Here, again, you are not assuming ‘The Null Hypothesis’, you are formulating specific null and alternate hypotheses in a specific situation. Whether you currently think the null or alternate hypotheses are correct is irrelevant to your safety though – if it’s coming towards you, whether it’s a thing with headlights or something else, it would be prudent to get out of the way. You can determine which of your hypotheses is correct as it passes.
I am intrigued to know where you are going with this. It started when I pointed out that you couldn’t apply some overall ‘Null Hypothesis’ idea to an overall question (Is there a Bigfoot) and you insisted I was wrong. Yet in your example, you apply a specific null hypothesis to a specific situation, which is the correct way to do it. Further, you now suggest I should break up one investigation into a subset of a dozen or more separate experiments, each with their own null hypotheses. That’s going too far the other way.
I could tell you how long I’ve been a scientist, how many degrees I have, how many years I spent researching and lecturing. I could tell you where to find definitions of the null hypothesis and other kinds of hypotheses, and how to detect (and avoid) the type 1 and type 2 errors associated with this approach. You will dismiss all that as ‘an appeal to authority’, ignore it, and continue to argue in circles.
Please, continue. I enjoy arguments.