Talk
Wednesday, May 10 |
3:27 PM
TRAnsparency CErtified (TRACE): Trusting Computational Research Without Repeating It
Abstract

Research communities across the natural and social sciences are increasingly concerned about the transparency and reproducibility of results obtained by computational means. Calls for increased transparency can be found in the policies of peer-reviewed journals and processing pipelines employed in the creation of research data products made available through science gateways, data portals, and statistical agencies. These communities recognize that the integrity of published results and data products is uncertain when it is not possible to trace their lineage or validate their production. Verifying the transparency or reproducibility of computational artifacts—by repeating computations and comparing results—is expensive, time-consuming, and difficult, and may be infeasible if the research products rely on resources that are subject to legitimate restrictions such as the use of sensitive or proprietary data; streaming, transient, or ephemeral data; and large-scale or specialized computational resources available only to approved or authorized users. The TRACE project is addressing this problem through an approach called certified transparency – a trustworthy record of computations signed by the systems within which they were performed. Using TRACE, system owners and operators certify the original execution of a computational workflow that produces findings or data products. By using a TRACE-enabled system, researchers produce transparent computational artifacts that no longer require verification, reducing burden on journal editors and reviewers seeking to ensure reproducibility and transparency of computational results. TRACE presents an innovative and efficient approach to ensuring the transparency of research that uses computational methods, is consistent with the vision outlined by the National Academies, and enables evidence-based policymaking based on transparent and trustworthy science.