Replicability in canine behavioral science

Lessons for conducting reliable research

Jeffrey R. Stevens

Canine Cognition and Human Interaction Lab
University of Nebraska-Lincoln
dogcog.unl.edu

jeffreyrstevens.quarto.pub/mit_2024

Canine behavioral science

Why study dogs

Dog roles

History of canine behavioral science

Challenges in canine behavioral science

  • Small sample sizes
  • Breed variation
  • Cultural variation in owners
  • Replication crisis

Replicating methods

Dog impulsivity

Owner perceptions of dog impulsivity

Research Question

Do owner perceptions of dog impulsivity match impulsivity in a behavioral task?

Based on and replicated Brady et al., 2018

Stevens et al. 2022, Animal Behavior & Cognition

Measuring owner perceptions

Dog Impulsivity Assessment Scale (DIAS)

  • Behavioral regulation
  • Aggression
  • Responsiveness

Wright et al., 2011

Stevens et al. 2022, Animal Behavior & Cognition

Measuring spatial impulsivity

Stevens et al. 2022, Animal Behavior & Cognition

Original results

Brady et al. 2018, Applied Anim Behav Sci

Our results

Stevens et al. 2022, Animal Behavior & Cognition

Study conclusions

  • Owners not good at predicting dog impulsivity

  • Did not replicate Brady et al., 2018

    • British know their dogs better than Americans
    • Different dog populations (breeds, neuter status)
    • Different owner populations
    • Less than robust methods in original study

Implications

  • Increase sample sizes
  • Conduct more replications (within and between labs)
  • Increase robustness and reproducibility of analyses
  • Post data publicly
  • Provide more and consistent info on owners
  • Consider populations of owners and dogs

Big team science

What is big team science?

Collaborations wherein a comparatively large number of researchers pool their intellectual and/or material resources to pursue a common goal

—Baumgartner et al., 2023 Royal Society Open Science.

Why big team science?

  • Larger sample sizes
  • Less effort per research site
  • Consensus in methods design
  • Diverse subject pool
  • Many analysts involved in analysis

ManyDogs Project

International research consortium of scientists with shared interest in canine behavior and cognition

ManyDogs Project Aims

  1. Attempt to replicate important studies
  2. Investigate moderators that require large sample sizes
  3. Develop consensus on methodological best practices
  4. Investigate cultural differences
  5. Set the bar for replicability of studies

ManyDogs 1

Research Question

Do dogs treat human pointing as communicative cues?

Results

ManyDogs Project et al. 2023, Animal Behavior & Cognition

Site effects

ManyDogs Project et al. 2023, Animal Behavior & Cognition

Site differences

ManyDogs Project et al. 2023, Animal Behavior & Cognition

Study conclusions

  • Dogs were slightly better than chance but no difference between conditions

  • Consistent findings across sites

  • Results do not match robust literature showing point following

  • Likely due to method choices

Implications

  • Again, consider populations of owners and dogs
  • Many analysts improved analysis
  • Diverse viewpoints can bias methods
  • Consider multi-site replication

Replicating interpretations

Dog quantity preferences

Research Question

What numerical factors contribute to dog quantity preferences?

What drives preference?

Numerical ratio (small / large) or difference (large - small)

Small Large Ratio Difference
1 2 0.50 1
2 4 0.50 2
1 3 0.33 2
2 6 0.33 4
Small Large Ratio Difference
1 2 0.50 1
2 4 0.50 2
1 3 0.33 2
2 6 0.33 4
Small Large Ratio Difference
1 2 0.50 1
2 4 0.50 2
1 3 0.33 2
2 6 0.33 4

Set-up

deBoer et al. PsyArXiv

Results

deBoer et al. PsyArXiv

Previous work on difference and ratio

Study Ratio effects Difference effects
Ward & Smuts 2007 Yes Yes
Baker et al. 2012 Yes Not tested
Miletto Petrazzini & Wynn 2016 Yes Not tested
Aulet et al. 2019 Yes Not tested
Rivas-Blanco et al. 2020 Yes Not tested

Study conclusions

  • Both difference and ratio account for numerical preference when analyzed separately

  • But only difference accounts when analyzed together

  • Revisit ubiquity of ratio effects

  • Include difference in models and design studies to test difference effects

Implications

  • Question dogma
  • Be open to alternative factors at play
  • Use strong inference and triangulation
  • Analyze other people’s data

Lessons Learned

Reliable science

Replicable

Reproducible

Robust

Open

Thank you!

Hunter DeBoer Hannah Fitzpatrick Anwyn Gatesy-Davis London Wolff ManyDogs Project

Hunter
DeBoer Hannah
Fitzpatrick Anwyn
Gatesy-Davis London
Wolff ManyDogs Project

jstevens5@unl.edu dogcog.unl.edu @unl_cchil