Reproducibility Assessment: Key Points

Code Inspection

Code inspection is the first step in reproducibility assessment. The inspection provides important information about the analytical workflow, computational requirements, and input data.
A main file contains essential code that facilitates reproducibility when more than one code file is required to repeat the analysis. The main file acts as the starting point in fully-automated push-button reproduction by running all code files in the proper sequence.
By provding the author’s contact information, computing requirements, and licensing terms, code header metadata signals to users of the author’s transparency and willingness to assist with reuse.
Non-executable comments in the code serve as a roadmap of the analytical workflow by identifying code segments and their purpose.

Executing code tests the reusability of the research compendium, which is a fundamental criterion of reproducibility.
Prior to running the code, the computing enviroment will need to meet the same or comparable techinical requirements as described in compendium documentation.
Simple errors that prevent code from fully executing can be easily addressed.

Every table, figure, and in-text number in the manuscript should be accounted for in code outputs.
The curating for reproducibilty workflow includes identifying specific commands in code files that produce outputs, and then verifying that reported results can be reproduced using that code.
Curators should document discrepanices found during the code output and manuscript review process and take steps to address them.