Journal Articles (Peer-Reviewed)
Blair, Courtney, Erica Gralla, Finley Wetmore, Jarrod Goentzel and Megan Peters (2021): A Systems Framework for International Development: The Data-Layered Causal Loop Diagram, Production and Operations Management.
Abstract: Meeting the United Nations Sustainable Development Goals (SDGs) will require adapting or redirecting a variety of very complex global and local human systems. It is essential that development scholars and practitioners have tools to understand the dynamics of these systems and the key drivers of their behavior, such as barriers to progress and leverage points for driving sustainable change. System dynamics tools are well suited to address this challenge, but they must first be adapted for the data-poor and fragmented environment of development work. Our key contribution is to extend the causal loop diagram (CLD) with a data layer that describes the status of and change in each variable based on available data. By testing dynamic hypotheses against the system’s actual behavior, it enables analysis of a system’s dynamics and behavioral drivers without simulation. The data-layered CLD was developed through a 4-year engagement with USAID/Uganda. Its contributions are illustrated through an application to agricultural financing in Uganda. Our analysis identified a lack of demand for agricultural loans as a major barrier to broadening agricultural financing, partially refuting an existing hypothesis that access to credit was the main constraint. Our work extends system dynamics theory to meet the challenges of this practice environment, enabling analysis of the complex dynamics that are crucial to achieving the SDGs.
Gralla, Erica, Paul T. Grogan, Jitesh H. Panchal and Zoe Szajnfarber (2020): A call for consensus on the use of representative model worlds in systems engineering and design, Systems Engineering, 23 (4): 436-442.
Abstract: Systems engineering and design (SE&D) researchers increasingly tackle questions at the intersection of technical and social aspects of complex systems design. Practical challenges of access, limited observation scope, and long timescales limit empirical study of SE&D phenomena. As a result, studies are typically conducted in model world settings abstracted from the real world, such as behavioral experiments with student subjects. Model worlds must be representative of the phenomena being studied to ensure insights generalize to the real-world settings. Currently, there is a lack of shared understanding and standards within the SE&D research community to evaluate representativeness of model worlds. This communication captures the results of ongoing efforts to build consensus on this topic: it defines the concept of model worlds, disambiguates representativeness from related concepts, and draws comparisons to other research domains. It outlines a potential path forward and calls for community participation in establishing shared standards for model world representativeness in SE&D research.
Chaudhari, Ashish M., Erica Gralla, Zoe Szajnfarber, Paul T. Grogan and Jitesh H. Panchal (2020): Designing Representative Model Worlds to Study Socio-Technical Phenomena: A Case Study of Communication Patterns in Engineering Systems Design, Journal of Mechanical Design, 142 (12): 121403-121420.
Abstract: The engineering of complex systems, such as aircraft and spacecraft, involves large number of individuals within multiple organizations spanning multiple years. Since it is challenging to perform empirical studies directly on real organizations at scale, some researchers in systems engineering and design have begun relying on abstracted model worlds that aim to be representative of the reference socio-technical system, but only preserve some aspects of it. However, there is a lack of corresponding knowledge on how to design representative model worlds for socio-technical research. Our objective is to create such knowledge through a reflective case study of the development of a model world. This “inner” study examines how two factors influence interdisciplinary communication during a concurrent design process. The reference real world system is a mission design laboratory (MDL) at NASA, and the model world is a simplified engine design problem in an undergraduate classroom environment. Our analysis focuses on the thought process followed, the key model world design decisions made, and a critical assessment of the extent to which communication phenomena in the model world (engine experiment) are representative of the real world (NASA’s MDL). We find that the engine experiment preserves some but not all of the communication patterns of interest, and we present case-specific lessons learned for achieving and increasing representativeness in this type of study. More generally, we find that representativeness depends not on matching subjects, tasks, and context separately, but rather on the behavior that emerges from the interplay of these three dimensions.
Moline, Julia, Jarrod Goentzel and Erica Gralla (2019): Approaches for Locating and Staffing FEMA´s Disaster Recovery Centers, Decision Sciences, 50 (5): 917-947.
Abstract: Disaster response is a challenging environment for data-driven decision support, because of its rarity, uniqueness, and high stakes. This article develops and demonstrates practical decision support approaches for an important problem faced by the U.S. Federal Emergency Management Agency (FEMA), identifies challenges and successes during their partial implementation, and explores how such approaches can encourage the adoption of data-driven decision-making in crisis situations. Specifically, this article develops approaches for locating and staffing temporary Disaster Recovery Centers (DRCs), which provide in-person service to disaster-affected communities. Working with FEMA, we developed two models that aim to improve service to survivors and minimize costs. One model fits easily into FEMA's current decision-making process, while the other further improves service by challenging some extant assumptions. By testing the models using data from three past disasters, we find that this decision support can result in cost savings of 75% on average by eliminating unnecessary DRCs and over-staffing and, at the same time, maintain or reduce the maximum travel time required for the disaster-affected population to access DRCs. Aspects of the models have already been used during disaster operations. FEMA's experience highlights the potential for data- and model-driven analyses to improve resource allocation and demonstrates an approach to improve organizational decisions by developing models that either align with or challenge the decision-making culture.
Herrmann, Jeffrey W., Michael Morency, Azrah Anparasan and Erica Gralla (2018): Evaluating Clustering Algorithms for Identifying Design Subproblems, Journal of Mechanical Design, 140 (8): 081401-081413.
Abstract: Understanding how humans decompose design problems will yield insights that can be applied to develop better support for human designers. However, there are few established methods for identifying the decompositions that human designers use. This paper discusses a method for identifying subproblems by analyzing when design variables were discussed concurrently by human designers. Four clustering techniques for grouping design variables were tested on a range of synthetic datasets designed to resemble data collected from design teams, and the accuracy of the clusters created by each algorithm was evaluated. A spectral clustering method was accurate for most problems and generally performed better than hierarchical (with Euclidean distance metric), Markov, or association rule clustering methods. The method's success should enable researchers to gain new insights into how human designers decompose complex design problems.
Reinker, Madison and Erica Gralla (2018): A Systems Dynamics Model of the Adoption of Improved Agricultural Inputs in Uganda, with Insights for Systems Approaches to Development, Systems, 6 (3).
Abstract: Designing international development projects is challenging because the complexity of the systems on which they act makes it difficult to identify the best leverage points for intervention. This paper seeks to identify the best combinations of interventions to increase the availability of and demand for quality seeds in Uganda and similar markets. A system dynamics model simulates the current dynamics in Ugandan seed markets based on data gathered by ongoing development projects. The findings show that one intervention is critical to enabling growth—investing in a system for verifying the quality of seeds—and that a combination of quality verification with education-oriented interventions is more powerful than quality verification alone. The results have implications for systems approaches to development: they suggest that a combination of interventions in different parts of the value chain enables larger changes than any single intervention alone.
Gralla, Erica, Jeffrey W. Herrmann and Michael Morency (2018): Design problem decomposition: an empirical study of small teams of facility designers, Research in Engineering Design, 30: 161-185.
Abstract: Decomposing complex design problems is an important component of design processes. When a design problem is too complex to solve all at once, the problem is decomposed into manageable subproblems. Previous work on design processes has identified some general decomposition patterns and has studied how individual designers decompose design problems; this study examines the way variables are grouped into subproblems, the process of decomposition, and whether small teams use similar decomposition patterns. Data were collected from five teams as they solved a facility design problem, and the subproblems that they considered were analyzed and compared. Using a mix of qualitative and quantitative analysis techniques, we examined (1) whether their subproblems group tightly coupled design variables (and separate weakly coupled variables); (2) whether their decompositions (subproblems and the sequence in which they were solved) follow a top–down design process; and (3) whether different teams used the same decompositions. Our results suggest that teams followed a partial top–down design process that moved from breadth- to depth-first search, and that subproblems were often driven by two types of coupling among design variables. However, the inconsistency of observed approaches suggests that there is room for improvement in how human designers decompose problems. By identifying these issues, the results lay a foundation for future research to provide better support for human designers in decomposing problems.
Gralla, Erica and Jarrod Goentzel (2018): Humanitarian transportation planning: Evaluation of practice-based heuristics and recommendations for improvement, European Journal of Operational Research, 269 (2): 436-450.
Abstract: Transportation bottlenecks are a common and critical problem in humanitarian response. There is a need for better planning and prioritization of vehicles to transport humanitarian aid to affected communities. Optimization approaches have been developed for transportation planning, but adoption has been limited, due in part to the difficulty of implementation. This paper develops the basis for an easily implementable decision support tool by building on current planning practices in the humanitarian sector. We draw on an observational study to describe current planning practices, then develop heuristic algorithms that represent the observed planning processes, and compare their solutions to each other and to those of a mixed-integer linear program. We identify key weaknesses to guide the development of more sophisticated heuristics or optimization models that fit with current planning practices. We also find that a simple practice-driven heuristic performs well when it prioritizes deliveries based on destination priority or distance, and we argue that automating such a heuristic in a decision support tool would maintain the simplicity and transparency to enable implementation in practice, and improve planning by saving time and increasing accuracy and consistency.
Greenberg, Brandon, Paule Voevodsky and Erica Gralla (2017): A capabilities-based framework for disaster response exercise design and evaluation: findings from oil spill response exercises, Journal of Homeland Security and Emergency Management, 13 (4).
Abstract: The responder community must be ready to respond quickly and effectively in the event of a disaster. In order to maintain readiness, many disaster response communities exercise their response capabilities on a regular basis. The critical challenge is to design, conduct, and evaluate exercises in a manner that effectively tests responders’ readiness and generates lessons that can improve readiness. This paper describes a framework to enable assessment of response readiness through evaluation of critical capabilities in exercises. It was developed for oil spill response based on the observation and analysis of four response exercises. The framework (1) identifies critical capabilities that lead to readiness for spill response, and maps them to (2) exercise design components that test each capability and (3) evaluation measures to evaluate each capability within an exercise. The framework enables continuous improvement by linking the evaluation of exercises to the critical capabilities required of an oil spill response organization; by evaluating the performance of specific capabilities, areas for improvement are clearly identified and can be re-tested in a future exercise. While the findings are necessarily specific to oil spill response, the principles apply to any disaster response context.
Szajnfarber, Zoe and Erica Gralla (2017): Qualitative methods for engineering systems: Why we need them and how to use them, Systems Engineering, 20 (6): 497-511.
Abstract: This paper discusses the role that qualitative methods can and should play in engineering systems research and lays out the process of doing good qualitative research. As engineering research increasingly focuses on sociotechnical systems, in which human behavior and organizational context play important roles in system behavior, there is an increasing need for the insights qualitative research can provide. This paper synthesizes the literature on qualitative methods and lessons from the authors’ experience employing qualitative methods to study a variety of engineering systems. We hope that by framing the key issues clearly, other engineers who hope to join the qualitative path will build on what we have learned so far to enable greater insight into engineering systems.
Gralla, Erica, Jarrod Goentzel and Charles Fine (2016): Problem Formulation and Solution Mechanisms: A Behavioral Study of Humanitarian Transportation Planning, Production and Operations Management, 25 (1): 22-35.
Abstract: When dealing with urgent, ill-defined problems, such as rapidly evolving emergency situations, operations managers have little time for problem formulation or solution. While the mechanisms by which humans formulate and solve problems have been described, mechanisms for rapid, concurrent formulating and solving are not well understood. This study investigates these mechanisms through a field study of transportation planning in a humanitarian response setting. The findings show that the problem is solved through greedy search and formulated through sensemaking, in which search enables updates to an evolving problem formulation, and the formulation directs and limits the search process. This study explores the implications of these findings for the development of better problem formulation processes and problem-solving strategies for urgent and ill-defined operations management problems.
Gralla, Erica and Zoe Szajnfarber (2016): Characterizing Representational Uncertainty in System Design and Operations, Systems Engineering, 19 (6): 535-548.
Abstract: This paper develops the concept of representational uncertainty to frame a critical challenge in systems engineering. Representational uncertainty arises in complex systems problems when the correct system representation cannot practically be known until some initial work has been undertaken. Drawing on empirical evidence from two very different system design problems, we illustrate the nature and prevalence of representational uncertainty in systems engineering practice. Our findings show that errors in the system representation may lead to wasted design work that explores the wrong tradespaces, expects the wrong value from design choices, and organizes work on the wrong set of decomposed subproblems. We find that mitigating representational uncertainty requires design processes that incorporate discovery of the system properties through a “reality check” early in the design process. We consider the implications for systems engineering processes and tools, and highlight directions for future research.
Gralla, Erica, Jarrod Goentzel and Bernard Chomilier (2015): Case study of a humanitarian logistics simulation exercise and insights for training design, Journal of Humanitarian Logistics and Supply Chain Management, 5 (1): 113-138.
Abstract: The purpose of this paper is to describe and analyze a successful training exercise in detail, through both a practical and a theoretical lens, in order to identify critical aspects of its success and enable others to build upon it; and to capture insights and lessons learned in a framework that will facilitate the design of future trainings for a variety of goals and audiences. The authors document and analyze the case study of a successful humanitarian logistics training exercise: the World Food Programme’s Logistics Response Team (WFP’s LRT) training. The LRT is described in detail in order to capture the extensive knowledge and experience that went into developing the full-scale, immersive exercise. The authors evaluate the LRT training through a theoretical lens, considering how it teaches the diverse set of skills required and identifying reasons for its success. The authors contrast the LRT with a light version developed for classroom use, and capture insights in a framework that highlights critical aspects of training design. The requirements and design aspects highlighted in the framework are very high level, but they focus attention on key aspects that should be considered. Future research should develop more targeted metrics for evaluating what people learn from training exercises. More generally, a systematic approach to capturing knowledge and codifying good practices should be developed. The detailed case study and framework provide a basis for the design and improvement of simulated emergency training exercises, which are common in the humanitarian practice community. The case study of WFP’s LRT training formally documents valuable knowledge and experience that went into its development. The humanitarian community can use the proposed framework to more systematically evaluate, improve, and extend training exercises.
Gralla, Erica, Jarrod Goentzel and Charles Fine (2014): Assessing Trade-offs among Multiple Objectives for Humanitarian Aid Delivery Using Expert Preferences, Production and Operations Management, 23 (6): 978-989.
Abstract: Humanitarian aid agencies deliver emergency supplies and services to people affected by disasters. Scholars and practitioners have developed modeling approaches to support aid delivery planning, but they have used objective functions with little validation as to the trade-offs among the multiple goals of aid delivery. We develop a method to value the performance of aid delivery plans based on expert preferences over five key attributes: the amount of cargo delivered, the prioritization of aid by commodity type, the prioritization of aid by delivery location, the speed of delivery, and the operational cost. Through a conjoint analysis survey, we measure the preferences of 18 experienced humanitarian logisticians. The survey results quantify the importance of each attribute and enable the development of a piecewise linear utility function that can be used as an objective function in optimization models. The results show that the amount of cargo delivered is the most valued objective and cost the least important. In addition, experts prioritize more vulnerable communities and more critical commodities, but not to the exclusion of others. With these insights and the experts’ utility functions, better humanitarian objective functions can be developed to enable better aid delivery in emergency response.
Gralla, Erica and de Weck, Olivier Ladislas (2007): Strategies for on-orbit assembly of modular spacecraft, Journal of the British Interplanetary Society, 60 (6): 219-227.
Abstract: Next-generation space exploration will most likely require the on-orbit assembly of large spacecraft. In order for any such exploration program to be sustainable, it must avoid the difficulties encountered by past programs by focusing on affordability. The advent of modular spacecraft potentially enables a variety of new, more affordable assembly techniques. This paper explores a number of on-orbit assembly methods for modular spacecraft, in order to understand the potential value of a reusable assembly support infrastructure. Four separate assembly strategies involving module self-assembly, tug-based assembly, and in-space refuelling are modelled and compared in terms of mass-to-orbit requirements for various on-orbit assembly tasks. Results show that reusable space tugs with in-space refuelling can reduce the required launch mass for on-orbit assembly between 5% and 41% (compared to self-assembly) for certain types of assembly tasks.