flowchart BT A["**Repeatability**<br/>Same researcher; <br/>same code/data"] --> B["**Reproducibility**<br/>Different researcher; <br/> same code/data"] B --> C["**Replicability**<br/>Different researcher; <br/> new code/data"] classDef r fill:#e3f2fd,stroke:#1e88e5,stroke-width:2px,color:#0d47a1, padding:50px, font-size:16px classDef p fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px,color:#1b5e20, padding:50px, font-size:16px classDef l fill:#fff8e1,stroke:#ef6c00,stroke-width:2px,color:#e65100, padding:50px, font-size:16px class A r class B p class C l
Repeatability, Reproducibility, and Replicability
Developing Transparent and Efficient Workflows
November 7, 2025
Overview
Background
Repeatability, reproducibility, and replicability
Why you should care
Tools to aid in better practices
Background
- transparency is lacking, social sciences are behind (especially crim), replication troubles, where do we go from here?
Source: The Turing Way
Focus on the controllables
What do we have control over?
Replicability
Distant; often not directly in our control
Reproducibility
Open data and code help reproducibility
Repeatability
Syntax based analyses; Clearly document research decisions
Why should you care?
Caution
How do I recreate figure 2?
Caution
Why does model 2 in table 3 have incorrect coefficients listed?
Caution
Why did I filter out those observations?
Caution
We are seeking additional information about the standard deviations for mean outcomes in Table 3 on page 7. Would you be able and willing to share those results?
Tools to help your research process
Repeatability
Stata
RStudio
Reproducibility
Github
- include video of local edits, pushed through desktop, to online repo
Going forward
What steps can I take now?
How does qualitative data fit into this?
What about sensitive data?
Thanks!
seth watts