If you are studying social sciences, you might be interested in this section about how to improve your statistics.
The purpose of testing a hypothesis is to see whether you can corroborate the theory that the hypothesis is derived from.
In the framework of Null Hypothesis Significance Testing, the null hypothesis mostly postulates that the difference between two groups is zero – an assumption that is not very realistic in many cases.
How to get around this problem, you ask? The answer is quite simple. What makes a theory more interesting and useful is the ability to predict an effect size from it. By this we mean the ability to quantify approximately how big the difference between certain groups will be. This is called point prediction (if a single value is predicted) or range prediction (if a range is given to approximate the exact value). It is also perfectly fine to not specify an upper boundary for the range, which would then be called a minimal effects test.
Using range predictions has three major advantages:
- You can make more specific predictions, which increases the verisimilitude of your theory. Take, for example, a horse race: You will get more money if you predict that the right horse will win the race, rather than predicting that ‘a horse’ will win the race (the latter is also not very cost-effective).
- You get clear cut-off points for falsifying your prediction: The effect you found either falls into your predicted range, or it doesn’t.
- Fewer irrelevant effects will be reported. Currently, small differences between groups that are significant due to a large sample size are reported in many scientific papers, simply because they are statistically significant. However, your predicted range of effect size will have to be theoretically relevant to be considered.
Some practicalities to consider:
- Setting the range for your prediction:
The lower value of the range prediction is equal to the smallest effect size you are interested in that is theoretically relevant. This is important when conducting an a priori power analysis (learn more about statistical power here) before your study. - Using equivalence tests:
The technique of equivalence tests (Lakens, Scheel, & Isager, 2018) can be used to specify equivalence ranges and then test if you can reject effects that fall outside of this range. - In early stages of research, you often cannot derive more specific predictions than that ‘something will happen’. And that is fine to start developing a theory. But you should aim to develop it into a theory that allows for deriving more specific predictions of effect sizes and respective ranges – because that is what makes a theory truly intriguing.
With these considerations in mind, you might want to rely less on two-sided hypothesis testing and rather think about the effect size and range that you want to corroborate or falsify in your study.
Sources:
http://daniellakens.blogspot.com/2018/07/strong-versus-weak-hypothesis-tests.html
Lakens, D., Scheel, A. M., & Isager, P. M.(2018).Equivalence Testing for Psychological Research: A Tutorial. Advances in Methods and Practices in Psychological Science, 2515245918770963. https://doi.org/10/gdj7s9
Need further help?
Click here to contact us