Summary of statistical survey results

My email to ecologists that participated in our survey of the statistical methods used by early-career ecologists:

Dear colleagues,

Too many months ago, you responded to a request to analyze a small dataset as part of a survey on statistical methods used by ecologists. We’re happy to say that the results of this small survey have now been published as part of a Forum “P values and model selection” in Ecology (http://www.esajournals.org/toc/ecol/95/3). Our paper was one of many great contributions to this Forum.

To cut to the chase: the focus of the Forum is an article by Paul Murtaugh “In defense of P values” showing that in a common statistical scenario, P values, delta AIC and confidence intervals are mathematically-interchangeable. From this result, Murtaugh argues that the case to “abandon P-values” in favor of AIC is over-stated. Given this context, we were interested in the extent to which ecologists defaulted (e.g. prior to peer review) to using null hypothesis significance testing (e.g. step-wise model selection) or information criterion (AIC) to select among many potentially-correlated predictors. Of 24 complete responses to our request, 10 participants used P values, 10 used AIC and 4 used alternative (e.g. Bayesian) approaches. Thus, not-withstanding over a decade of calling for the abandoment of P values, they still are the primary choice of many ecologists. Of greater concern, there was considerable variation in the predictors included in the final models by survey participants, even when using similar approaches.

Thanks again for your contribution to this project, and we hope that you find them to read the Forum.

Sincerely,

John Stanton-Geddes, Cintia Gomes de Freitas and Cristian Dambros