In science, the success of an experiment is often determined by a measure called “statistical significance.” A result is considered to be “significant” if the difference observed in the experiment between groups (of people, plants, etc) would be very unlikely if no difference actually exists. The common cutoff for “very unlikely” is that you’d see a difference as big or bigger only 5 percent of the time if it wasn’t really there — a cutoff that might seem, at first sight, very strict.
Statistical significance has been used to draw a bright line between experimental success and failure. Achieving an experimental result with statistical significance often determines if a scientist’s paper gets published or if further research gets funded. That makes the measure far too important in deciding research priorities, statisticians say, and so it’s time to throw it in the trash.
More than 800 statisticians and scientists are calling for an end to judging studies by statistical significance in a comment published in Nature. An accompanying special issue of the American Statistician makes the manifesto crystal clear in its introduction: “‘statistically significant’ — don’t say it and don’t use it.” Science and statistics have never been so simple as to cater to convenient cutoffs. A P value, no matter how small, is just a probability. It doesn’t mean an experiment worked. And it doesn’t tell you if the difference in results between experimental groups is big or small. In fact, it doesn’t even say whether the difference is meaningful.
There is good reason to want to scrap statistical significance. But with so much research now built around the concept, it’s unclear how — or with what other measures — the scientific community could replace it. (1)
All science is based on the probability that something will happen.
But it is the improbable things which shape the cosmos.
The birth of the universe.
The death of a God.
The genesis of a human being…
People living for the wrong things.
People dying for the right ones.
Yes, it is a paradoxical world. Not a sane one.
Stop looking at the probable things.
Look at the clouds. Feel the sun behind them.
There is nothing more dull than the things which happen.
Predicting everything. Knowing it all.
The cornerstone of our civilization.
The cause of our demise.
Can you ever predict anything without knowing everything?
Can you ever know anything without already experiencing everything?
Start looking at the things which will never happen.
And right when you won’t be looking…
You will know they already did!
