Introduction

Overview

Teaching: 0 min
Exercises: 0 min

Questions

• Key question

Objectives

• Understand the current issues and barriers to reproducibility

• Understand why the whole workflow is important

• Understand documentation/organizational issues underpinning this

What is the problem?

Researchers often have three broad goals for their work:

• they are interested in studying innovative ideas; they want to discover new things about how the world works
• they are interested in producing reproducible results - they want to be able to find their results again, and they want others to be able to find them as well
• they want to build off of their own and others’ work; in order to accumulate knowledge over time, researchers want to advance our understanding of studies and innovation
• over the past few years there has been a growing concern that many of the findings in published literature may not replicate.

There is evidence from a broad range of fields, including cancer biology, psychology, and political science, that the published literature may not be as reliable.

This workshop will introduce standard research practices that lead scientists to produce research that is more difficult to reproduce and replicate, thus leading to generally low levels of replicability.

What is the Reproducibility?

• Computational reproducibility - to take a copy of someone else’s data and their code/analysis scripts, rerunning their exact methods, you would be able to reproduce the numbers that are reported in their results.
• Empirical reproducibility - reproducing what was done in a study. Having enough information to rerun the experiment or survey the way it was originally conducted.
• Replicability - Having enough information to reproduce a study’s protocol completely, and reproduce the analyses completely, and run them on independent data set, would we reproduce the results? Would we come to the same statistical conclusions as the original study?

Often in science we want the results of studies to be reproducible, so we want our scientific findings to replicate.

What will we learn in this workshop?

• Work with the Open Science Framework (OSF) to increase the documentation and transparency of your workflow, and learn tools that will help you to implement these changes.
• To do this, we’re all going to be working through a hypothetical research study.
• You’ll work on building an open, transparent research project from start to finish in order to learn good data/project management practices and use of the OSF.

Getting Started

• Group participants in 2 or 3 people per group

• Setup a new account or sign in to OSF here

Key Points

• First key point.