Is psychology going to the dogs?

What canine science can tell us about reliable research

Jeffrey R. Stevens

Canine Cognition and Human Interaction Lab
University of Nebraska-Lincoln
dogcog.unl.edu

jeffreyrstevens.quarto.pub/nps2025

Replication crisis

A crisis in psychology

Manuscript info for Bem 2011 Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect

Manuscript info for Wagenmakers et al. 2011 Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi: Comment on Bem (2011)

Manuscript info for Simmons et al. 2011 False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant

Replication in psychology

Colored boxes showing number of replication studies whose outcomes matched the original's results. 39 studies matched and 61 studies did not. Colors represent the degree to which the replication results were similar to the original results.

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science. doi:10.1126/science.aac4716

Causes of the crisis

  • Small sample sizes
  • Optional stopping
  • Poor understanding and use of statistics
  • P-hacking
  • HARKing
  • Inconsistency in methods
  • Bias against replications
  • WEIRD populations

A push for more reliable research

Manuscript info for Stevens 2017: Replicability and Reproducibility in Comparative Psychology.

Canine Cognition and Human Interaction Lab

Brown dog approaching cup on floor with researcher in background. Logo for Canine Cognition and Human Interaction Lab Brown dog being petted by undergraduate student.

Black sign reading Established 2018.

Replicating methods

Dog impulsivity

Brown dog chasing cormorant on the beach.

Brown dog rolling around on grass.

Brown dog licking treats in a jar.

Photos by: Pete Godfrey, Michael Oxendine, Tamas Pap on Unsplash

Owner perceptions of dog impulsivity

Research Question

Do owner perceptions of dog impulsivity match impulsivity in a behavioral task?

Replicated Brady et al., 2018

Stevens et al. 2022, Animal Behavior & Cognition

Measuring owner perceptions

Dog Impulsivity Assessment Scale (DIAS)

  • I would consider my dog to be very impulsive
  • My dog does not think before it acts
  • My dog appears to have a lot of control over how it responds
  • My dog is not very patient

Wright et al., 2011

Stevens et al. 2022, Animal Behavior & Cognition

Measuring spatial impulsivity

Brown dog looking at plate of three treats next to plate of one treat.

Stevens et al. 2022, Animal Behavior & Cognition

Original results

Three panel plot of Maximum Distance Traveled (in meters) as a function of Owner Perceptions of Impulsivity (DIAS) for Adult Study 1, Adult Study 2, and Pup Study. There are no data.

Three panel plot of Maximum Distance Traveled (in meters) as a function of Owner Perceptions of Impulsivity (DIAS) for Adult Study 1, Adult Study 2, and Pup Study. Data show a negative relationship for Adulty Study 1 and 2 but no relationship for Pup Study.

Brady et al. 2018, Applied Anim Behav Sci

Our results

Scatterplot of Distance traveled (in meters) as a function of Owner Perception of Impulsivity. There are no data.

Scatterplot of Distance traveled (in meters) as a function of Owner Perception of Impulsivity. Data show no relationship.

Two scatterplots of Distance traveled (in meters) as a function of Owner Perception of Impulsivity. Data show no relationship in both plots.

Stevens et al. 2022, Animal Behavior & Cognition

Owners not good at predicting dog impulsivity

Conclusions

Did not replicate Brady et al., 2018

  • British know their dogs better than Nebraskans?
  • Different owner populations?
    • British vs. American dog-owner bonds
    • Demographics
  • Different dog populations?
    • Breeds
    • Neuter status

Implications for reliability

  • Increase sample sizes
  • Conduct more replications (within and between labs)
  • Increase robustness and reproducibility of analyses
  • Post data publicly
  • Provide more and consistent info on owners
  • Consider populations of owners and dogs

Aggregating studies

Impulsivity as a trait in domestic dogs

Research Question

Is impulsivity a behavioral trait in dogs?

  • Do dogs’ impulsivity levels correlate across tasks?
  • Do dogs’ impulsivity levels correlate with owner perceptions?

Barela et al. 2024, Journal of Comparative Psychology

Meta-analysis

  • Systematic review of the literature to extract and compare statistics from all published studies
  • Searched 9,900 records from Scopus, Web of Science, and PsychINFO
  • 13 articles conducted correlations between impulsivity tasks and/or between the Dog Impulsivity Assessment Scale (DIAS) and behavioral tasks

Barela et al. 2024, Journal of Comparative Psychology

Task examples

Barela et al. 2024, Journal of Comparative Psychology

Systematic review: task pairs

Barela et al. 2024, Journal of Comparative Psychology

Systematic review: DIAS and tasks

Barela et al. 2024, Journal of Comparative Psychology

Impulsivity does not seem to be a behavioral trait

Implications for reliability

  • Enhance statistical power
  • Gives larger picture of effects
  • Help resolve conflicting results
  • Reduces biases toward positive effects

Big team science

What is big team science?

Collaborations wherein a comparatively large number of researchers pool their intellectual and/or material resources to pursue a common goal

—Baumgartner et al., 2023 Royal Society Open Science.

World map with pins showing locations of Psychological Science Accelerator research sites.

ManyDogs Project

International research consortium of scientists with shared interest in canine behavior and cognition

ManyDogs logo

manydogsproject.github.io

World map with pins showing locations of ManyDogs Project research sites

ManyDogs 1

Research Question

Do dogs treat human pointing as communicative cues?

Black dog watching experimenter point to one of two blue cups.

ManyDogs Project et al. 2023, Animal Behavior & Cognition

Dogs follow the point

Plot of Accuracy as a function of Condition (Non-communicative and Communicative). There are no data.

Spaghetti plot of Accuracy as a function of Condition (Non-communicative and Communicative). Data show no discernable pattern.

Spaghetti plot of Accuracy as a function of Condition (Non-communicative and Communicative) with group means and 95% confidence intervals overlaid. Means are slightly greater than chance (0.5) but not different from each other.

ManyDogs Project et al. 2023, Animal Behavior & Cognition

Sites differ in owner age distribution

Panel of 17 histograms (one for each research site) showing frequency of participant owner ages across sites. Sites show marked variation with some uniformly distributed, some normally distributed, and some skewed to younger ages.

ManyDogs Project et al. 2023, Animal Behavior & Cognition

Dogs followed pointing but not communicatively

Implications for reliability

  • Again, consider populations of owners and dogs
  • Many analysts improved analysis
  • Consider multi-site replication

Lessons Learned

Reliable science

Illustration of two scientists replicating each other's chemistry experiment.

Replicable

Illustration of person walking through windy trail of reproduciblity, including data, tools, code, and results.

Reproducible

Illustration of large global network with people all over the world connected to one another.

Robust

Open access logo

Open

What can you do?

Close-up image of brown dog's face.

Thank you!

Research assistants

  • Jessica Barela
  • Megan Herridge
  • Kylie Hughes-Duvall
  • Maddie Mathias
  • London Wolff
  • Yasmin Worth
  • McKenna Yohe

ManyDogs logo National Science Foundation logo

Contact

jstevens5@unl.edu

jeffreyrstevens.quarto.pub/nps2025


Canine Cognition and Human Interaction Lab

dogcog.unl.edu

unl_cchil

unl.cchil

@unlcchil.bsky.social