Collings, R. D., Eaton, L. G., Foley, J. T., & Nelson, A. J.
Using dyadic proportion tests with individual pairwise comparisons to overcome power limitations with small-sample experiments
The dyadic proportions test (DPT) is an analytic approach intended for small sample research (e.g., clinical and exploratory studies). The procedure involves using proportion tests to compare outcomes from dyads (one participant in each condition) associated with the actual experimental assignment with outcomes from all possible dyadic assignments. Monte Carlo analyses in Study 1 revealed that sampling distributions generated by DPTs were normally-distributed, as expected with traditional approaches. In Study 2, data from eight small-sample experiments (n = 8) were analyzed with DPTs and t tests. DPTs provided more power and resulted in more significant effects (p < .05) than did t tests. In Study 3, data from an intervention study conducted with a small clinical sample (n = 12) were analyzed with DPTs and t tests. The experimental effect was significant with the DPT, but not the t test. The DPT is an appropriate approach when large samples are not feasible.
Testing, Psychometrics, Methodology in Applied Psychology, 2019, Vol. 26, pp. 221-236, DOI: 10.4473/TPM26.2.4
