Effective strategies for multi-sectoral research using large-scale models

Abstract

Large-scale integrated assessment models (IAMs) have become critical knowledge objects and tools for global-scope simulation of energy, economic, engineering, and environmental systems. IAMs are widely used in assessment of climate change mitigation (for instance, within the IPCC) and other Sustainable Development Goals (SDGs). In order to sharpen the relevance of insights from large-scale modeling, researchers often link them into multi-model frameworks together with detailed sectoral (sub-)models of, for instance, transport and mobility; the building stock; critical materials; or water use—or with models of narrower scope (e.g. national or sub-national) but finer resolution. These links capture feedbacks and interactions that may strongly shape transitions to comprehensive sustainability.

This presentation begins with current expectations that guide IAM-based research. In particular, it is no longer satisfactory that individual models and 1-to-1 frameworks can analyse, for instance, (a) possible recoveries from the COVID-19 pandemic; (b) the long-term evolution of transport and mobility; and (c) the energy and climate impacts of changing materials flows. Increasingly, stakeholders require assessments that are both integrated and highly detailed in multiple sectors/areas such as these. Teams are challenged to produce this knowledge while also progressing in openness, transparency, reproducibility, and validity of research.

I argue that meeting these challenges requires systems researchers and modelers to acknowledge that they are engaged in processes of collaborative software development: models are, in an important sense, also complex, long-lived, and evolving pieces of software. This perspective motivates the use of strategies that are common in professional software development, yet woefully underutilized in academia and research: standardization, testing, modularity and reuse, an emphasis on documentation, and iterative workflows with short cycle times.

Using the example of the MESSAGEix-GLOBIOM family of models developed by the IIASA Energy, Climate, and Environment (ECE) Program, I demonstrate how adopting best practices of development translates to better science that uses people and resources more efficiently. For instance, the practice of unit-testing code is also a scientific strategy for ensuring internal validity: when code implements research methods, then automated tests ensure that those methods are correct representations of the phenomena represented. This in turn avoids exhausting researchers' time in performing manual validity checks. Next, the practice of modularity entails clearly-defined interfaces: in multi-sectoral or cross-domain integrated assessment frameworks, this is achieved through clear definitions of the background concepts, measures, scales, and system boundaries. Making these items explicit helps researchers see quickly and precisely what translation is required to make (sub-)models interoperable with one another.

Finally, sustainability requires the very research processes that support the transition to be equitable and inclusive. I explain how the practices of making software free and open source, writing complete documentation, and continuous development lower barriers to understanding and participating in IAM-based research, especially for those not privileged to work at the few institutions that have the resources to maintain large-scale models. As well, a broad user base translates to a flow of contributions, improved model quality, and ultimately improved perceptions of legitimacy for modeling work.