Working in a Reproducible & Replicable Manner

Increasing Documentation

Documentation is a necessary part of reproducibility and replicability in research, however, many research projects include little or no detail on workflow to allow others to reproduce the results of a study. Considerations for creating useful documentation include:

• Version Control – Version control is a system of recording changes made to a file or set of files over time that can be later reviewed. Version control allows researchers to track when changes were made and document the reasoning behind those decisions. The upcoming module, Open Software, offers more guidance on version control.
• Including appropriate meta information – Meta information describes the content and structure of a project, data, and files within it. README files help make projects more accessible, where codebooks are intended as a self-explanatory guide to variables in a data file. These types of documentation are helpful to both the research team and those interested in replicating or reproducing the study.
• File organization and sharing – Storing all materials related to the research study in one place, where they are organized in a meaningful way, help make sure all materials are accounted for when building readme files or workflow documentation.

The most important factor in workflows for reproducibility and replicability is access to the materials that make up the research such as the tools used for creating the data (e.g., research instruments such as surveys), scripts for manipulation and analysis of the data, a record of the decisions about the specific tools and methods used, and the data used or collected. Sharing all the elements of the research project in one location with persistent identifiers (e.g., Digital Object Identifiers (DOIs)) provides the opportunity for the researchers to engage in open workflows while supporting the scientific ideal of reproducible results.

We will further explore documentation and organization in the next part of this module, Best Practices for Organizing.

Reflections

Think back on a project, research or otherwise, from two years ago. Do you remember the details of the project and how you completed it? Do you remember why you made the decisions you did around the project? Could you tell someone how to complete the project themselves with the same results?

Even if you have a very good memory, the answer is probably no.