Participatory Modeling (PM) is becoming increasingly common in environmental planning and conservation due to advances in cyberinfrastructure and to greater recognition of the importance of engaging a diverse array of stakeholders in decision making. We initiated this reflective article at the first of a series of workshops on PM, sponsored by the National Socio-Environmental Synthesis Center (SESYNC). The goal of the workshop series is to organize and consolidate scholarship around the practice of PM. PM encompasses the use of a broad range of modeling approaches in various forms of collaboration among practitioners, academics, and other stakeholders who engage in a purposeful learning process that elicits and formalizes the implicit and explicit knowledge of participants to support decision-making and action. Our knowledge community has learned important lessons from their experiences, successes, and failures. Given our experience, we firmly believe that PM has the potential to help communities guide themselves toward more positive futures. We offer these lessons learned in the hope that they will guide practitioners as they assist communities with using meaningful, appropriate, and illuminating modeling tools in participatory processes.
The reflections in this paper emerge from the authors’ over 200 years of cumulative and diverse experience conducting PM processes with communities concerned with different issues. Some examples of our work can be found in Table 1, at www.participatorymodeling.org, and in Gray et al., 2018.
|Address flooding using decentralized green infrastructure||Zellner et al., In Press|
|Ensure the sustainability of groundwater supplies||Zellner et al., 2012|
|Protect biodiversity and ecosystem functions, and understand drivers of bushmeat trade||Nayaki et al., 2014|
|Model the social and economic impacts of climate change on coastal resources||Gray et al., 2014|
|Provide decision-support for wildlife managers in the Pacific||Htun et al., 2016|
|Support conservation planning in relation to invasive species management||Gray et al., 2017|
|Enact with farmers water resource planning at times of drought||Douglas et al., 2016|
|Address land use conflicts and manage trade-offs of a range of ecosystem services||Hubacek et al., 2009; Schmitt Olabisi et al., 2017|
Our collective interdisciplinary background allowed for reflection on a rich range of collaborative modeling approaches, incorporating insights and experiences with environmental modeling, spatial analysis, urban and regional planning, psychology, anthropology, computer science, social science, and economics. The reflections in this paper are also informed by prior summary and review articles. These prior articles, covering multiple participatory studies, describe commonalities, articulate different overarching structures for the PM processes, describe lessons learned, and propose principles for effective PM (e.g., Voinov and Brown Gaddis, 2008; Voinov and Bousquet, 2010; Oteros-Rozas et al., 2015; Voinov et al., 2016; Gray et al., 2018). This paper focuses less on summarizing previous PM projects, and more on what we have learned about the challenges of implementing PM, especially in creating meaningful stakeholder and modeler partnerships within the complex social and political contexts where PM is most needed.
These reflections and lessons learned were generated and organized as follows. During our first workshop (February 2016), each participant wrote a summary of the key lessons that they had learned in their PM research and practice. We used inductive logic (Charmaz and Belgrave, 2007) to identify patterns arising from compiled data and grouped them into the three key themes described below. We built on this synthesis through several iterations, drawing from diverse literature to provide the foundation for this paper.
We have not previously shared these lessons widely because their articulation is frequently extraneous to traditional academic scholarship. We are often discouraged from publishing about failures (Becu et al., 2007). Our intent herein is to share our experience beyond advancing the technological dimension of modeling. We seek to encourage, and maybe even inspire others to embrace the uncertainty and messiness inherent to PM through lessons we have learned via many trials, successes, and even more errors, very much like any modeling process (Railsback and Grimm, 2012). It is our hope that sharing these lessons will help other practitioners skip some of the more painful learning steps we ourselves have worked through, and more effectively build the powerful collective knowledge and social capital that can emerge from PM processes.
Figure 1 illustrates a common evolution of assumptions and practices in PM, where researchers, who are eager to put their skills and knowledge to good use in supporting decisions, move from a strictly technical perspective towards full embrace of the partnership perspective. Each puzzle piece depicted in Figure 1 is necessary but not sufficient in describing how best to achieve innovative solution-building and action. For example, a common initial assumption is that providing decision-makers with the “right” information (e.g., more accurate/precise data, an understanding of interaction effects) and the “right” tools (e.g., models that accurately represent processes, interactions, and provide meaningful results), is sufficient for them to solve their problems. This is not enough to lead to improved decisions, actions, and effective solutions, possibly because decision-makers distrust modeling processes and tools that exclude their involvement.
Faced with falling short of their goals, researchers add increasingly more nuance to their approaches, moving next to an emphasis on communicating in the “right” ways (e.g., clearly translating and presenting information and models to both decision-makers and stakeholders), and when that too fails, moving towards more inclusion and participation. Researchers may assume that engaging selected stakeholders in data collection, analysis, and modeling will facilitate ownership and make stakeholders feel empowered and informed. When anticipated outcomes do not occur, researchers then turn to the need for diversity to enhance innovation and representation. Conflict may arise in engaging multiple stakeholders. In response, while keeping in mind previous assumptions and responses, the researchers focus on social processes, towards an understanding that attending to emotions, transparency of models and assumptions, power dynamics, and tradeoffs are key elements of effective PM.
This evolution takes place in individuals as well as across whole fields where initial technocratic approaches are enhanced by and eventually embrace more inclusive approaches. We have seen similar evolutions in risk management (Fischhoff, 1995), in probabilistic decision analysis (Spetzler et al., 2016), and in stakeholder engagement (Sterling et al., 2017). Recognition of greater complexity leads to consideration of processes as well as outcomes; more inclusive public input and consultation approaches, including possibilities for civic engagement and co-management (Leong et al., 2009, 2012); and a “commitment to action” based on the entire process (Leong et al., 2011; Spetzler et al., 2016).
We have grouped our reflections into three main categories regarding lessons learned: a) how modelers need to engage in PM, b) how to adapt to the social and political context of collaboration, and c) how to set up the PM process itself. Our reflections may be of most interest to modelers and researchers but we also share these lessons learned for facilitators and communities who may be interested in working in this arena.
“Participatory modelers,” in the context of this paper, are individuals who facilitate, organize, and develop models for PM projects. A diverse set of skills, including prior content knowledge, facilitation skills, and technical modeling skills are needed to ensure that the PM process and resulting models are useful and helpful to the participating communities. It is a rare individual who possesses all of these skills; it is often better to consider a collaborative team as “the” participatory modeler (Prell et al., 2007). Here we report the skills that we have found to be most important for the team:
Stakeholder groups are not monolithic; they have varying organizational structures, multiple and competing objectives, and they often evolve over time. Individual stakeholders cannot represent an entire group; stakeholders and stakeholder groups are dynamic and cross-scalar. Multiple manifestations of a “community” exist within any particular geographic region. Communities can be defined, for instance, by political, religious, genealogical/familial/clan, or by resource management oversight characteristics. Setting manageable boundaries for selection of groups to participate in a modeling initiative, and identifying in which stages different stakeholders should participate, is crucial (Prell et al., 2007; Reed et al., 2013a, 2013b, 2017). Some stakeholders need to be engaged from the very beginning (Reed et al., 2009; Cormier-Salem, 2014; Sterling et al., 2017). Yet, trying to engage every stakeholder may mean spending a lot of time reaching consensus but not creating a lot of clarity on action (Büscher and de Beer, 2011; Sterling et al., 2017). Experience, context, and clear stakeholder engagement strategies help to overcome these challenges (Sterling et al., 2017).
Navigating power asymmetries is a tricky ethical and logistical issue. On the one hand, when significant power differentials exist, inviting all stakeholders (a “neutral posture”) may simply exacerbate initial power asymmetries if less powerful participants are less able to contribute (Kritek, 2002). On the other hand, excluding or separating the more powerful participants (a “non-neutral posture”) to empower the weaker actors engenders questions regarding how people are chosen (Barnaud and Van Paassen, 2013) and potentially weakens the usefulness of the results of the PM process.
Understanding the social and political context within which one is operating is key. For instance, some stakeholders may have specific expectations about roles and relationships based on previous experiences. Others may have little or no experience in working with modelers on planning and decision-making, and may need extra assistance in learning how to participate effectively. Still others may have engagement fatigue. Governance structures differ around the world, and those structures and norms may change the real and perceived benefits of participating in a PM process. Some groups may be uncomfortable with group decision-making, preferring to defer to figures of authority. And, at times, groups are set on the decisions that they want to make, and, while they may be open to a collaborative engagement, they are resistant to new ideas. Our lessons from and about operating within a variety of social and political contexts include:
Managing the participation process in PM is as important as managing the model building process. Models within a collaborative process can function as “boundary objects” (Star and Griesemer, 1989; Harvey and Chrisman, 1998), providing a means to bridge ideas across disciplines and participants’ perspectives, and thus promoting learning through collaboration (Akkerman and Bakker, 2011; Zellner et al., 2012). PM processes help stakeholders bring to light assumptions, causes, solutions, and values of which they may have only subconscious awareness. This provides opportunities for users to make ideas visible and open for discussion, negotiation, and revision, and supports constructive discourse. PM models also allow individual and collective cognition to be externalized and made explicit, by mentally offloading difficult tasks into an environment (e.g., computer screens and notebooks) where thinking can be organized and discussed (Bart, 1995; Zellner, 2008). Furthermore, because modeling forces us to explicitly formalize diverse knowledges, ideally individuals coming from different backgrounds should be able to communicate in this shared workspace (van der Leeuw, 2004).
Robust, meaningful, and impactful PM is a huge effort, especially if compared to a more traditional top-down or authoritative decision-making process. It takes time, funding, and iterative engagement to build the relationships and trust among the participants, tools, and process (particularly how to navigate conflict and tradeoffs), in addition to building the models themselves. That said, once the iterative process is established, further collective innovation and action may come more easily and quickly. Participants become used to thinking collaboratively with modeling tools. Slowing down to engage in such processes paradoxically gets communities to where they want to be, faster (Zellner and Campbell, In Press).
Previous experiences with collaborative processes shape participant expectations of PM, however. Community trust in such processes can be eroded if conveners of a past collaborative process did not take their input seriously, or if members are afraid to voice their opinions, perspectives, and knowledge in public. Some processes become driven more by the modelers and by the needs of a quantitative model than the interests and needs of the community. This can establish an expectation that the modelers are “in charge” and lead to less innovation and participation from the community in model development and use.
Lessons we have learned about how to manage these issues include:
PM is a powerful approach for addressing complex social and environmental problems. While it holds great promise, it can also come at a high cost: it is difficult to gather adequate skills, funds, and participants’ time, and it takes time to gradually build strong relationships between scientists, community partners, and public agencies. It may require that actors with decision-making responsibilities cede some of their authority to a group and process over which they have little control. Based on the lessons presented above, we conclude our reflection with some cross-cutting recommendations.
Members of a project team should reflect on their abilities with regards to our lessons learned on the role of participatory modelers. All participatory modelers need to be aware of how to support relationship building among participants, and design and use tools accordingly. If critical skills are missing, the project team should first build its own capacity and may consider recruiting project partners with the needed expertise. Successful PM efforts create an inclusive environment that supports participants with differing values, ideas, and priorities. Sharing (time, viewpoints, stories, common challenges, food, small slices of life, etc.) helps to develop these relationships and understanding. Stakeholders know more about the problem, obstacles and opportunities, and the community than do modelers from outside the community. Provide time for people to understand the purpose of the modeling, but more importantly, for participants to share their wants and needs. Modelers are a support to them, to help them achieve their goals and aspirations. Find ways to “check in” regularly and in different ways to see how learning is progressing in individuals and across the collaboration, and how stakeholders are perceiving the collaboration and the issues in general (Bennett, 2016). This includes structured observations of the dialogue and deliberations occurring around the PM tools, and what actions participants have collectively agreed to support. Power dynamics frequently influence how people from different backgrounds interact. Try to understand the motivations underlying participants’ behavior, and explicitly acknowledge and legitimize those motivations. Note that researchers are all part of the process, not outsiders taking objective notes from afar, so modelers should also expose their motivations and reflect on their biases.
Whatever tools and techniques are used, make sure to work with a subset of the stakeholders to design the PM with them. Not only will the PM process (tools, location and time, facilitation and deliberation setups) thus be comfortable to users, but also the non-modeler participants become advocates and facilitators of these tools, building trust in the process, and easing the transition towards self-reliance and appropriate tool use to inform decision-making. Provide feedback to stakeholders on model development and, to groups who were not involved in its development, on prior model use and outcomes. Recruit and, if necessary, train different kinds of facilitators (technical/modeling, community) to help manage the social interactions among community participants, between modelers and participants, and between users and the tools developed and used (Hovmand, 2014).
Consider how to deal with dissonance between what participants initially expect and what the model suggests; if the gap is too wide, it can be difficult for participants to embrace the results and make effective decisions. This discomfort may be compounded by the fact that uncertainty is inherent in any complex problem that is the focus of the PM activity, and is often reflected in the modeling process and outcomes. Participants may equate such uncertainty to ignorance, and thus distrust the modeling effort and its insights, instead of incorporating the uncertainty into the design of robust policies that can be effective in an uncertain world (Zellner 2008). Scaffolding between expectations and results needs to be built into the PM process, in order to address confirmatory bias (Hoch et al., 2015; Zellner et al., In Press).
Keep models simple, relevant, and tractable, distilled to key decision points of importance to the participants (Zellner et al., In Press). The purpose of PM is for stakeholders to participate in the analysis of the complex issues they face, to gain insights relative to their roles in problems and in their solutions, to harness their collective creativity in designing solutions, and to examine and deliberate about the various tradeoffs with appropriate information. If models become intractable or irrelevant “black boxes” that do not adequately represent and make it easy for stakeholders to examine assumptions, goals, and values, then that possibility is lost.
Finally, PM involves the community in understanding and modeling what are typically complex human, social, and natural systems. Although a project may “wrap up,” achieving its decision-specific goals, the community and those complex systems continue to exist and evolve. A valuable result of PM would be to foster ongoing dialogue and collaborative analysis that is adaptive to the inevitable surprises brought about by complex problems and systems. Theoretical frameworks, case studies, and individual experiences with PM and other forms of collaborative governance are signaling what some have identified as a “new governance era” (e.g., Leong et al., 2011). Yet, these approaches are not fully integrated into agency policies and practices (Leong et al., 2011; Zellner and Campbell, 2015) despite notable international efforts and commitments such as the Local Agenda 21 coming out of the UN Rio Declaration on Environment and Development, the European Union’s (EU) Aarhus Convention on Access to Information, Public Participation in Decision-Making and Access to Justice in Environmental Matters, and the EU’s Water Framework Directive. Having overcome the initial hurdles of setting up a PM process, it is worth investing in collaborative stakeholder networks to facilitate ongoing learning and exchanges from peer to peer, outside of PM efforts. Long-term relationships with stakeholders can carry across many complex projects, transferring the ways of thinking across projects and building the capacity to effectively deal with such complexity in different cases.
Our hope is that these lessons are useful to new practitioners who venture on this path, with a realistic view of what PM entails, recommendations of steps to take to ensure a productive process, and the enthusiasm and confidence in this approach as a support for a new form of collaborative governance to grapple with difficult and persistent problems.
Co-authors of this paper are U.S. Government employees, therefore: Any use of trade, product, or firm names in this publication is for descriptive purposes only and does not imply endorsement by the U.S. Government. We thank Nadav Gazit for comments, technical edits, and graphic design, Don Rosenberry and Collin Lawrence for U.S. Geological Survey technical reviews, and Amanda Sigouin for editing support. We appreciate the helpful comments from two outside reviewers and the thoughtful guidance from the editors.
This work was supported by the National Socio-Environmental Synthesis Center (SESYNC) under funding received from the National Science Foundation DBI-1052875. The material is also based upon work supported by the National Science Foundation (NSF) under Grants No. EF-1427091, 1427453, and 1444184. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. In addition, this research was conducted by the Assessing Biocultural Indicators Working Group supported in part by SNAPP: Science for Nature and People Partnership, a collaboration of The Nature Conservancy, the Wildlife Conservation Society and the National Center for Ecological Analysis and Synthesis (NCEAS) at the University of California, Santa Barbara.
The authors have no competing interests to declare.
Contributed to conception and design: EJS, MZ, KEJ, KL; Contributed to acquisition of data: All authors; Contributed to analysis and interpretation of data: EJS, MZ, KEJ, KL, PDG; Drafted and/or revised the article: All authors; Approved the submitted version for publication: EJS, MZ.
Akkerman, SF and Bakker, A. 2011. Boundary crossing and boundary objects. Rev Educ Res 81(2): 132–169. DOI: 10.3102/0034654311404435
Anderson, CA, Lepper, MR and Ross, L. 1980. Perseverance of social theories: The role of explanation in the persistence of discredited information. J Pers Soc Psychol 39(6): 1037. DOI: 10.1037/h0077720
Barnaud, C, Le Page, C, Dumrongrojwatthana, P and Trébuil, G. 2013. Spatial representations are not neutral: Lessons from a participatory agent-based modelling process in a land-use conflict. Environmental Modelling & Software 45: 150–159. DOI: 10.1016/j.envsoft.2011.11.016
Barnaud, C and Van Paassen, A. 2013. Equity, power games, and legitimacy: Dilemmas of participatory natural resource management. Ecology and Society 18(2): 21. DOI: 10.5751/ES-05459-180221
Barreteau, O, Antona, M, D’Aquino, P, Aubert, S, Boissau, S, Bousquet, F, Daré, W, Etienne, M, Le Page, C and Mathevet, R. 2003. Our companion modelling approach. JASSS-J Artif Soc S 6(1): n.p.
Bart, J. 1995. Acceptance criteria for using individual-based models to make management decisions. Ecol Appl 5(2): 411–420. DOI: 10.2307/1942032
Becu, N, Neef, A, Schreinemachers, P and Sangkapitux, C. 2007. Participatory computer simulation to support collective decision-making: Potential and limits of stakeholder involvement. Land Use Policy 25(4): 498–509. DOI: 10.1016/j.landusepol.2007.11.002
Bennett, NJ. 2016. Using perceptions as evidence to improve conservation and environmental management. Conserv Biol 30(3): 582–592. DOI: 10.1111/cobi.12681
Borshchev, A and Filippov, A. 2004. From system dynamics and discrete event to practical agent based modeling: Reasons, techniques, tools. The 22nd International Conference of the System Dynamics Society, July 25–29. Oxford, England.
Büscher, B and de Beer, E. 2011. The contemporary paradox of long-term planning for social-ecological change and its effects on the discourse-practice divide: evidence from Southern Africa. J Environ Plann Man 54(3): 301–318. DOI: 10.1080/09640568.2010.506075
Cash, DW, Clark, WC, Alcock, F, Dickson, NM, Eckley, N, Guston, DH, Jäger, J and Mitchell, RB. 2003. Knowledge systems for sustainable development. Proceedings of the national academy of sciences 100(14): 8086–8091. DOI: 10.1073/pnas.1231332100
Charmaz, K and Belgrave, LL. 2007. Grounded Theory. In: Ritzer, G (ed.), The blackwell encyclopedia of sociology 4: 25. Oxford, UK: John Wiley & Sons, Ltd. DOI: 10.1002/9781405165518.wbeosg070.pub2
Conley, A and Moote, MA. 2003. Evaluating collaborative natural resource management. Society and Natural Resources 16(5): 371–386. DOI: 10.1080/08941920309181
Cormier-Salem, M-C. 2014. Participatory governance of Marine Protected Areas: A political challenge, an ethical imperative, different trajectories. Senegal case studies. SAPIENS 7(2).
Douglas, EM, Wheeler, SA, Smith, DJ, Overton, IC, Gray, SA, Doody, TM and Crossman, ND. 2016. Using mental-modelling to explore how irrigators in the Murray–Darling Basin make water-use decisions. J Hydrol: Reg Stud 6: 1–12. DOI: 10.1016/j.ejrh.2016.01.035
FAO. 2016. Free Prior and Informed Consent: An indigenous peoples’ right and a good practice for local communities. Manual for Project Practitioners . FAO. Available at: http://www.fao.org/3/a-i6190e.pdf.
Fischhoff, B. 1995. Risk perception and communication unplugged: Twenty years of process. Risk Analysis 15(2): 137–145. DOI: 10.1111/j.1539-6924.1995.tb00308.x
Forsyth, DR. 2014. Group Dynamics . 6th ed. Belmont, CA: Wadsworth Cengage Learning 34.
Glynn, PD. 2014. W(h)ither the oracle? Cognitive biases and other human challenges of integrated environmental modeling. In: Ames, DP, Quinn, NWT and Rizzoli, AE (eds.), 7th Intl Congress on Env Modelling and Software . San Diego, USA: International Environmental Modelling and Software Society.
Glynn, PD. 2017. Integrated environmental modelling: Human decisions, human challenges. Geol Soc London Spec Publ 408(1): 161–182. DOI: 10.1144/SP408.9
Glynn, PD, Voinov, AA, Shapiro, CD and White, PA. 2017. From data to decisions: Processing information, biases, and beliefs for improved management of natural resources and environments. Earth’s Future 5(4): 356–378. DOI: 10.1002/2016EF000487
Gray, S, Gagnon, A, Gray, S, O’Dwyer, B, O’Mahony, C, Muir, D, Devoy, RJN, Falaleeva, M and Gault, J. 2014. Are coastal managers detecting the problem? Assessing stakeholder perception of climate vulnerability using Fuzzy Cognitive Mapping. Ocean Coast Manage 94: 74–89. DOI: 10.1016/j.ocecoaman.2013.11.008
Gray, S, Jordan, R, Crall, A, Newman, G, Hmelo-Silver, C, Huang, J, Novak, W, Mellor, D, Frensley, T, Prysby, M and Singer, A. 2017. Combining participatory modelling and citizen science to support volunteer conservation action. Biol Conserv 208: 76–86. DOI: 10.1016/j.biocon.2016.07.037
Gray, S, Voinov, A, Paolisso, M, Jordan, R, BenDor, T, Bommel, P, Glynn, P, Hedelin, B, Hubacek, K, Introne, J, Kolagani, N, Laursen, B, Prell, C, Schmitt Olabisi, L, Singer, A, Sterling, E and Zellner, M. 2018. Purpose, processes, partnerships, and products: Four Ps to advance participatory socio-environmental modeling. Ecol Appl 28(1): 46–61. DOI: 10.1002/eap.1627
Hämäläinen, RP. 2015. Behavioural issues in environmental modelling–The missing perspective. Environ Modell Softw 73: 244–253. DOI: 10.1016/j.envsoft.2015.08.019
Hämäläinen, RP and Alaja, S. 2008. The threat of weighting biases in environmental decision analysis. Ecol Econ 68(1–2): 556–569. DOI: 10.1016/j.ecolecon.2008.05.025
Harvey, F and Chrisman, N. 1998. Boundary objects and the social construction of GIS technology. Environ Plann A 30(9): 1683–1694. DOI: 10.1068/a301683
Hoch, C, Zellner, M, Milz, D, Radinsky, J and Lyons, L. 2015. Seeing is not believing: Cognitive bias and modelling in collaborative planning. Plan Theor Pract 16(3): 319–335. DOI: 10.1080/14649357.2015.1045015
Hovmand, PS. 2014. Community based system dynamics . 1st ed. New York, USA: Springer-Verlag. DOI: 10.1007/978-1-4614-8763-0
Htun, H, Gray, SA, Lepczyk, CA, Titmus, A and Adams, K. 2016. Combining watershed models and knowledge-based models to predict local-scale impacts of climate change on endangered wildlife. Environ Modell Softw 84: 440–457. DOI: 10.1016/j.envsoft.2016.07.009
Hubacek, K, Beharry, N, Bonn, A, Burt, T, Holden, J, Ravera, F, Reed, M, Stringer, L and Tarrasón, D. 2009. Ecosystem services in dynamic and contested landscapes: The case of UK uplands. In: Winter, M, Lobley, M (eds.), What is land for? The Food, Fuel and Climate Change Debate , 167–188. London, UK: Earthscan.
Kaner, S. 2014. Facilitator’s Guide to Participatory Decision-Making , 3rd ed. Hoboken, New Jersey: Jossey-Bass. 363. (Jossey-Bass Business & Management Series).
Kritek, PB. 2002. Negotiating at an uneven table: Developing moral courage in resolving our conflicts . 2nd ed. Hoboken, New Jersey: Jossey-Bass. (Jossey-Bass Health Series).
Kruger, J and Dunning, D. 1999. Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J Pers Soc Psychol 77(6): 1121. DOI: 10.1037/0022-3522.214.171.1241
Leong, KM, Decker, DJ, Lauber, TB and Chase, L. 2012. Stakeholders as beneficiaries of wildlife management. In: Decker, DJ, Riley, SJ and Siemer, WF (eds.), Human dimensions of wildlife management , 2nd ed., 26–41. Baltimore, USA: The Johns Hopkins University Press.
Leong, KM, Decker, DJ, Lauber, TB, Raik, DB and Siemer, WF. 2009. Overcoming jurisdictional boundaries through stakeholder engagement and collaborative governance: Lessons learned from white-tailed deer management in the U.S. In: Andersson, K, Eklund, E, Lehtola, M and Salmi, P (eds.), Beyond the rural-urban divide: Cross-continental perspectives on the differentiated countryside and its regulation 360: 221–247. Bingley, UK: Emerald Publishing Group.
Leong, KM, Emmerson, DP and Byron, R. 2011. The new governance era: Implications for collaborative conservation and adaptive management in department of the interior agencies. Hum Dimens Wildl 16(4): 236–243. DOI: 10.1080/10871209.2011.585436
Madden, F and McQuinn, B. 2014. Conservation’s blind spot: The case for conflict transformation in wildlife conservation. Biol Conserv 178: 97–106. DOI: 10.1016/j.biocon.2014.07.015
Martin, R and Schlüter, M. 2015. Combining system dynamics and agent-based modeling to analyze social-ecological interactions—An example from modeling restoration of a shallow lake. Front Environ Sci 3(66): 1–15. DOI: 10.3389/fenvs.2015.00066
Matsui, K. 2015. Problems of defining and validating traditional knowledge: A historical approach. The International Indigenous Policy Journal 6(2). Available at: https://ir.lib.uwo.ca/iipj/vol6/iss2/2. DOI: 10.18584/iipj.2015.6.2.2
McCarter, J, Sterling, E, Jupiter, S, Cullman, G, Albert, S, Basi, M, Betley, E, Boseto, D, Bulehite, E, Harron, R, Holland, P, Horning, N, Hughes, A, Jino, N, Malone, C, Mauli, S, Pae, B, Papae, R, Rence, F, Revo, O, Taqala, E, Taqu, M, Woltz, H and Filardi, CE. 2018. Biocultural approaches to developing well-being indicators in Solomon Islands. Ecology and Society 23(1). DOI: 10.5751/ES-09867-230132
Milz, D, Zellner, M, Hoch, C, Radinsky, J, Pudlock, K and Lyons, L. 2017. Reconsidering scale: Using GIS to inform spatial planning talk. Planning Practice & Research , 1–18.
Minter, T, van der Ploeg, J, Pedrablanca, M, Sunderland, T and Persoon, GA. 2014. Limits to indigenous participation: The Agta and the northern sierra Madre Natural Park, the Philippines. Hum Ecol 42(5): 769–778. DOI: 10.1007/s10745-014-9673-5
Morewedge, CK, Yoon, H, Scopelliti, I, Symborski, CW, Korris, JH and Kassam, K. 2015. Debiasing decisions: Improved decision making with a single training intervention. Policy Insights from the Behavioral and Brain Sciences 2(1): 129–140. DOI: 10.1177/2372732215600886
Nayaki, A, Gray, S, Lepczyk, C, Skibins, J and Rentsch, D. 2014. Local-scale dynamics and local drivers of bushmeat trade. Conserv Biol 28(5): 1403–1414. DOI: 10.1111/cobi.12316
Nickerson, RS. 1998. Confirmation bias: A ubiquitous phenomenon in many guises. Rev Gen Psychol 2(2): 175. DOI: 10.1037/1089-26126.96.36.199
Oteros-Rozas, E, Martín-López, B, Daw, TM, Bohensky, EL, Butler, JR, Hill, R, Martin-Ortega, J, Quinlan, A, Ravera, F, Ruiz-Mallén, I, Thyresson, M, Mistry, J, Palomo, I, Peterson, GD, Plieninger, T, Waylen, KA, Beach, D, Bohnet, IC, Hamann, M, Hanspach, J, Hubacek, K, Lavorel, S and Vilardy, S. 2015. Participatory scenario planning in place-based social-ecological research: Insights and experiences from 23 case studies. Ecol Soc 20(4): 32. DOI: 10.5751/ES-07985-200432
Pink, D. 2006. A whole new mind: Why right-brainers will rule the future. New York, USA: Penguin Group.
Prell, C, Hubacek, K, Reed, M, Quinn, C, Jin, N, Holden, J, Burt, T, Kirby, M and Sendzimir, J. 2007. If you have a hammer everything looks like a nail: Traditional versus participatory model building. Interdiscipl Sci Rev 32(3): 263–282. DOI: 10.1179/030801807X211720
Radinsky, J, Milz, D, Zellner, M, Pudlock, K, Witek, C and Lyons, L. 2017. How planners and stakeholders learn with visualization tools: Using learning sciences methods to examine planning processes. J Environ Plann Man 60(7): 1296–1323. DOI: 10.1080/09640568.2016.1221795
Railsback, SF and Grimm, V. 2012. Agent-based and individual-based modeling: A practical introduction. Princeton, USA: Princeton University Press.
Reed, MS, Graves, A, Dandy, N, Posthumus, H, Hubacek, K, Morris, J, Prell, C, Quinn, CH and Stringer, LC. 2009. Who’s in and why? A typology of stakeholder analysis methods for natural resource management. J Environ Manage 90(5): 1933–1949. DOI: 10.1016/j.jenvman.2009.01.001
Reed, MS, Hubacek, K, Bonn, A, Burt, TP, Holden, J, Stringer, LC, Beharry-Borg, N, Buckmaster, S, Chapman, D, Chapman, PJ, Clay, GD, Cornell, SJ, Dougill, AJ, Evely, AC, Fraser, EDG, Jin, N, Irvine, BJ, Kirkby, MJ, Kunin, WE, Prell, C, Quinn, CH, Slee, B, Stagl, S, Termansen, M, Thorp, S and Worrall, F. 2013a. Anticipating and managing future trade-offs and complementarities between ecosystem services. Ecol Soc 18(1): 5. DOI: 10.5751/ES-04924-180105
Reed, MS, Kenter, J, Bonn, A, Broad, K, Burt, T, Fazey, IR, Fraser, EDG, Hubacek, K, Nainggolan, D, Quinn, CH, Stringer, LC and Ravera, F. 2013b. Participatory scenario development for environmental management: A methodological framework illustrated with experience from the UK uplands. J Environ Manage 128: 345–362. DOI: 10.1016/j.jenvman.2013.05.016
Reed, MS, Vella, S, Challies, E, de Vente, J, Frewer, L, Hohenwallner-Ries, D, Huber, T, Neumann, RK, Oughton, EA, del Ceno, JS and van Delden, H. 2017. A theory of participation: What makes stakeholder and public engagement in environmental management work? Restor Ecol . DOI: 10.1111/rec.12541
Reid, R, Nkedianye, D, Said, M, Kaelo, D, Neselle, M, Makui, O, Onetu, L, Kiruswa, S, Ole Kamuaro, N, Kristjanson, P, Ogutu, J, BurnSilver, SB, Goldman, MJ, Boone, RB, Galvin, KA, Dickson, NM and Clark, WC. 2016. Evolution of models to support community and policy action with science: Balancing pastoral livelihoods and wildlife conservation in savannas of East Africa. P Natl Acad Sci USA 113(17): 4579–4584. DOI: 10.1073/pnas.0900313106
Schmitt Olabisi, L, Liverpool-Tasie, S, Rivers, L, Ligmann-Zielinska, A, Du, J, Denny, R, Marquart-Pyatt, S and Sidibé, A. 2017. Using participatory modeling processes to identify sources of climate risk in West Africa. Environment Systems and Decisions 32: 23. DOI: 10.1007/s10669-017-9653-6
Sébastien, L and Bauler, T. 2013. Use and influence of composite indicators for sustainable development at the EU-level. Ecol Indic 35: 3–12. DOI: 10.1016/j.ecolind.2013.04.014
Simpson, JA, Farrell, AK, Orina, MM and Rothman, AJ. 2015. Power and social influence in relationships. In: Mikulincer, M and Shaver, PR (eds.), APA Handbook of personality and social psychology: Interpersonal relations 3. Washington, DC, USA: American Psychological Association. DOI: 10.1037/14344-015
Singer, A, Jetter, A, Ellsworth, L, Gray, S, Zhang, P and Oussama Laraichi, M. 2017. Policy scenarios for fire-adapted communities: Understanding stakeholder risk perceptions in Ashland, Oregon. Project Report . Portland, USA: Portland State University. Available at: www.mentalmodeling.org.
Spetzler, C, Winter, H and Meyer, J. 2016. Decision quality: Value creation from better business decisions. Hoboken, USA: John Wiley & Sons. DOI: 10.1002/9781119176657
Spiegelhalter, D, Pearson, M and Short, I. 2011. Visualizing uncertainty about the future. Science 333(6048): 1393–1400. DOI: 10.1126/science.1191181
Star, SL and Griesemer, JR. 1989. Institutional ecology, translations’ and boundary objects: Amateurs and professionals in Berkeley’s Museum of Vertebrate Zoology, 1907–39. Soc Stud Sci 19(3): 387–420. DOI: 10.1177/030631289019003001
Sterling, EJ, Betley, E, Sigouin, A, Gomez, A, Toomey, A, Cullman, G, Malone, C, Pekor, A, Arengo, F, Blair, M, Filardi, C, Landrigan, K and Porzecanski, AL. 2017. Assessing the evidence for stakeholder engagement in biodiversity conservation. Biol Conserv 209: 159–171. DOI: 10.1016/j.biocon.2017.02.008
Sturm, RE and Antonakis, J. 2015. Interpersonal power: A review, critique, and research agenda. J Manag 41(1): 136–163. DOI: 10.1177/0149206314555769
Susskind, L, McKearnan, S and Thomas-Larmer, J. 1999. The Consensus Building Handbook: A Comprehensive Guide to Reaching Agreement. Thousand Oaks, USA: SAGE Publication Ltd.
Tengö, M, Brondizio, ES, Elmqvist, T, Malmer, P and Spierenburg, M. 2014. Connecting diverse knowledge systems for enhanced ecosystem governance: The multiple evidence base approach. Ambio 43(5): 579–591. DOI: 10.1007/s13280-014-0501-3
Tversky, A and Kahneman, D. 1974. Judgment under uncertainty: Heuristics and biases. Science 185(4157): 1124–1131. DOI: 10.1126/science.185.4157.1124
Van Der Leeuw, SE. 2004. Why model? Cybernet Syst 35(2–3): 117–128. DOI: 10.1080/01969720490426803
van Kerkhoff, L. 2013. Knowledge governance for sustainable development: A review. Challenges in Sustainability 1(2): 82–93. DOI: 10.12924/cis2013.01020082
Voinov, A and Bousquet, F. 2010. Modelling with stakeholders. Environ Modell Softw 25(11): 1268–1281. DOI: 10.1016/j.envsoft.2010.03.007
Voinov, A and Brown Gaddis, EJ. 2008. Lessons for successful participatory watershed modeling: A perspective from modeling practitioners. Ecol Model 216(2): 197–207. DOI: 10.1016/j.ecolmodel.2008.03.010
Voinov, A, Kolagani, N, McCall, MK, Glynn, PD, Kragt, ME, Ostermann, FO, Pierce, SA and Ramu, P. 2016. Modelling with stakeholders–next generation. Environ Modell Softw 77: 196–220. DOI: 10.1016/j.envsoft.2015.11.016
Voinov, A, Seppelt, R, Reis, S, Nabel, JE and Shokravi, S. 2014. Values in socio-environmental modelling: Persuasion for action or excuse for inaction. Environ Modell Softw 53: 207–212. DOI: 10.1016/j.envsoft.2013.12.005
Wu, K and Dunning, D. 2017. Hypocognition: Making sense of the landscape beyond one’s conceptual reach. Rev Gen Psychol 1(2): 25–35. DOI: 10.1037/gpr0000126
Wondolleck, JM and Yaffee, SL. 2000. Making collaboration work: Lessons from innovation in natural resource management . Washington, DC, USA: Island Press.
Zellner, ML. 2008. Embracing complexity and uncertainty: The potential of agent-based modeling for environmental planning and policy. Plan Theor Pract 9(4): 437–457. DOI: 10.1080/14649350802481470
Zellner, ML and Campbell, SD. 2015. Planning for deep-rooted problems: What can we learn from aligning complex systems and wicked problems? Plan Theor Pract 16(4): 457–478. DOI: 10.1080/14649357.2015.1084360
Zellner, ML and Campbell, S. n.d. Planning with(in) complexity: Pathways to extend collaborative planning, incremental planning, and Big Data with complex systems modelling. In: de Roo, G, Yamu, C and Zuidem, C (eds.), Research handbook on planning and complexity . Cheltenham, UK: Edward Elgar Publishing. In Press.
Zellner, ML, Keller, J and Massey, D. 2018. Robbins Renewal and Resiliency Project Management. Report to the Chicago Community Trust, Chicago, IL: University of Illinois at Chicago.
Zellner, ML, Lyons, LB, Hoch, CJ, Weizeorick, J, Kunda, C and Milz, DC. 2012. Modeling, learning, and planning together: An application of participatory agent-based modeling to environmental planning. URISA Journal 24(1): 77–93.
Zellner, ML, Lyons, L, Milz, D, Shelley, J, Hoch, C, Massey, D and Radinsky, J. n.d. Participatory complex systems modeling for environmental planning: Opportunities and barriers to learning and policy innovation. In: Porter, WF, Zhao, J, Schmitt Olabisi, L and McNall, M (eds.), Innovations in collaborative modeling . East Lansing, USA: Michigan State University Press. In Press.