Laypeople, policymakers, and practitioners often seek evidence-based recommendations for complex decision-making about health, education, and work. Although books and reviews are available to them, these resources tend to oversimplify results and omit contextual details. In contrast, Evidence Synthesis Systems (ESS) like evidence maps and evidence databases synthesize and reuse evidence by summarizing large literature bases, forming holistic judgments, and assessing generalizability. They aim to help diverse users make practical decisions with confidence. Unfortunately, ESS authors currently lack an understanding of how to synthesize and communicate evidence best against the backdrop of thorny philosophical, methodological, and interaction design problems. To that end, we are weaving insights from the philosophy of science, theories from the psychological sciences, and advances from metascience to steer the design and use of ESS.