Forschung

Participatory foresight for technology assessment

Towards an evaluation approach for knowledge co-creation

Mahshid Sotoudeh, Institute of Technology Assessment (ITA) of the Austrian Academy of Sciences (ÖAW), 1030 Vienna, Apostelgasse 23 (msotoud@oeaw.ac.at)

Niklas Gudowsky, Institute of Technology Assessment (ITA) of the Austrian Academy of Sciences (ÖAW) (niklas.gudowsky@oeaw.ac.at)

Technology assessment (TA) frequently uses forward-looking methods to anticipate socio-technical changes and their corresponding implications to deduce advice for policy and society. In recent years, participatory methods have increasingly been applied to identify the expectations of society towards future technologies. In this context, several TA projects have developed, applied and adapted a participatory foresight method to engage citizens as well as other actor groups into co-generating advice for research and innovation agenda setting in a standardized process; namely, the multi-perspective and multi-step CIVISTI method (Citizens’ Visions on Science, Technology and Innovation). Over the course of the past ten years, about 560 lay citizens without specialised knowledge on technology and innovation and 610 experts and stakeholders have taken part in these processes of co-generation of knowledge. In this contribution, we use our experience with this method and elaborate some criteria for the evaluation of knowledge co-generation and mutual learning in participatory foresight processes within TA.

Partizipative Foresight für die Technikfolgenabschätzung

Hin zu einem evaluationsgestützen Ansatz für die Wissens-ko-Generierung

Technikfolgenabschätzung (TA) verwendet häufig vorausschauende Methoden, um sozio-technische Entwicklungen und deren Auswirkungen zu antizipieren und daraus Empfehlungen für Politik und Gesellschaft abzuleiten. In den letzten Jahren wurden zunehmend partizipative Methoden eingesetzt, um die Erwartungen der Gesellschaft bezüglich Zukunftstechnologien zu identifizieren. In diesem Zusammenhang haben mehrere TA-Projekte eine partizipative Foresight-Methode entwickelt, angewandt und angepasst, um Bürgerinnen und Bürger sowie andere Akteursgruppen über einen standardisierten Prozess in die Mitgestaltung von Forschungs- und Innovationsagenden einzubeziehen: die multiperspektivische und mehrstufige CIVISTI-Methode (Citizens’ Visions on Science, Technology and Innovation). Im Laufe der letzten zehn Jahre haben etwa 560 Laien ohne spezielle Kenntnisse über Technologie und Innovation und 610 Fachleute an diesen Prozessen der Wissens-ko-generierung teilgenommen. In diesem Beitrag nutzen wir unsere Erfahrungen mit dieser Methode, um Kriterien für die Bewertung von Wissens-ko-Generierung und wechselseitigem Lernen in partizipativen Foresight-Prozessen innerhalb der TA abzuleiten.

Keywords: technology assessment, participatory foresight, co-creation of knowledge, mutual learning, evaluation criteria

This is an article distributed under the terms of the Creative Commons Attribution License CCBY 4.0 (https://creativecommons.org/licenses/by/4.0/)

TATuP Bd. 27 Nr. 2 (2018), S. 53–59, https://doi.org/10.14512/tatup.27.2.53

Submitted: 04. 12. 2017. Peer reviewed. Accepted: 22. 03. 2018

Introduction and background

From the perspective of futurists and foresight practitioners, participatory foresight is seen as one possible way to provide a greater variety of perspectives and ensure certain “knowledge encounters” within the foresight process (Nikolova 2014). TA, however, has developed participatory foresight methods to assess different expectations of future technologies for the shaping of technology and innovation policy. Polk and Knutsson (2008, p. 644) argue in favour of “[…] establishing trans-disciplinary platforms for knowledge production. Goals and visions are jointly created via participation and mutual learning, rather than by assigning science and technology the burden of producing the certainty that is demanded by political and economic rationality”. Here, there is a focus on multi-actor governance by posing the question of what targets future socio-technical developments and thus their framework conditions should be aimed at. Therefore, participatory foresight in TA is increasingly oriented towards processes of knowledge co-generation between different actor groups.

The terms co-creation, co-production and co-generation are often used synonymously in literature in various fields, while being closely connected to concepts of public engagement, open innovation or inclusive governance (Regeer and Bunders 2009; Voorberg et al. 2015; Ramírez and García-Peñalvo 2018). Ostrom et al. (1978), for instance, conducted pioneering work with regard to co-production of public services, and today co-creation is considered “a cornerstone for social innovation in the public sector” (Voorberg et al. 2015, p. 1346). In this contribution, we will consider knowledge co-generation within participatory foresight in TA with regard to enabling actors to “develop a shared repertoire of socially robust knowledge” (van Veen et al. 2013, p. 105). With regard to CIVISTI-processes, this shared repertoire can be found in their final results, e. g. the newsletter from the future (Gudowsky et al. 2017). Such results interlink intermediate products that are based on different types of knowledge, e. g. citizens’ visions, results from analysis of visions regarding values and needs, experts’ and stakeholders’ recommendations or their validation and prioritization through citizens (Gudowsky and Sotoudeh 2017). While citizens’ visions remain in their original state, all subsequent steps not only refer to them, but built upon them and thus a transformation of knowledge takes place. The optimization of these special participatory processes for foresight in TA and the evaluation processes should therefore consider not only the mechanisms of information exchange and communication between society, science and policy in participatory TA (pTA) or quality factors such as fairness, transparency and efficiency for internal and external legitimacy of processes and results (Rower and Frewer 2005; Joss and Bellucci 2002), but also the knowledge exchange and generation in a multi-actor foresight processes. With regard to CIVISTI, different types of knowledge are elicited in different steps of the process, and are interlinked and transformed through various analytical and participatory steps of the method (Jacobi et al. 2010; Gudowsky and Sotoudeh 2017). Engaged actors in the CIVISTI cases discussed in this contribution remained spatially and timely separated, and thus one could argue that no true co-creation took place. However, the resulting final products display co-generated knowledge as stated above. Here, a systematic insight into the co-generation of knowledge in participatory foresight methods could improve the structuring of inter- and transdisciplinary work and actively promote the uptake of results into technology and innovation policy. Finally, we will suggest an extension to the evaluation of participatory foresight processes for TA for optimization of the co-generation process.

For strategic planning, it is of utmost importance to have separate steps for creative process of visioning and pragmatic feasibility thinking.

In order to analyse the co-generation process, we need a distinction between different types of knowledge and their interactions during co-generation. A detailed analysis of these processes is beyond the scope of this contribution. We present an analysis based on an initial selection of different knowledge types for co-generation and distinguish between three categories for the characterization of knowledge in participatory foresight in TA:

  1. In the first category, actor-specific knowledge is pointed out. The evaluation process should be sensitive to interactions of actor-specific views and perspectives during the co-generation of knowledge. Nowotny (2003) has emphasised the need for generating socially robust knowledge and bringing together the many different knowledge types and dimensions (see also Pohl and Hirsch-Hadorn 2006). This category addresses who contributes to co-generation and how different perspectives are involved.
  2. The second category supports distinctions between different types of knowledge for decision-making by participatory foresight for technology policy. For this analysis we distinguish between four relevant types of knowledge within this category: cognitive knowledge (e. g. based on different actors’ perspectives on scientific and technological issues for the analysis of arguments and decision making) and pragmatic dimensions of knowledge based on political culture and the question of power to influence decision-making processes. Cognitive and pragmatic knowledge are components of scientific knowledge (Kuznetsov et al. 2012, p. 880). Knowledge of normative values (rooted in social norms, cultural identity of participants, etc.) is relevant for this analysis, in the context of transdisciplinarity. Scientific analysis for the multi-dimensional characterisation of potential impacts of suggested future solutions should be linked with normative knowledge as a prerequisite for determining the acceptability of solutions. Renn (2009, p. 565) describes the need for normative orientation knowledge for risk management for a sustainable development. The relevance to distinguish cognitive, normative and pragmatic dimensions in decision making procedures in pTA has been highlighted with regard to “inequality”[1] in Belluci et al. (2002, p. 22). For our purposes, we apply this threefold distinction to different dimensions of knowledge. For an analysis of knowledge co-generation within participatory foresight in TA, we consider also “emotional knowledge” based on hopes and concerns of individuals: “Emotional knowledge is created by emotions and integrated together with cognitive knowledge into our mental representation of the world.” (Brătianu and Orzea 2014, p. 43)
  3. In the third category, long-term planning in participatory foresight in TA is framed through systemic knowledge, target knowledge and transformation knowledge. Systemic knowledge (knowledge about what is) is knowledge of the complex interrelationships of present problems of everyday life on a social, ecological and economic level. Target knowledge is about evaluation and description of future state (knowledge about what should be and what should not be). It is knowledge about how standards can be justified. Transformation knowledge is about how to get from the existing to the target state and how the transition from the current to the target state is to be designed and implemented (see Dubielzig und Schaltegger 2004).

We will show the role of different knowledge categories on an example, namely the CIVISTI method (Citizens’ Visions on Science, Technology and Innovation), since the process of knowledge co-generation was taken into account in the design of the CIVISTI method. The method has been developed and applied on regional, national as well as supra-national (EU) levels since 2008 to co-generate knowledge on societally relevant issues for target setting for research and innovation policies and programmes. The process of communication and deliberation as well as results are published and provide our knowledge pool for this analysis of co-generation of knowledge within participatory foresight in TA (see Sotoudeh et al. 2014; Gudowsky et al. 2012; Gudowsky and Peissl 2016; Gudowsky and Sotoudeh 2017; Gudowsky et al. 2017). The next section provides an overview of the transdisciplinary foresight method CIVISTI.

CIVISTI Citizens’ visioning workshops (step 1)

CIVISTI Stakeholders’ and experts’ workshops (step2)

Framing role for long-term planning (knowledge category c)

Co-generation of target knowledge with citizen participation

Using systemic and transformation knowledge for co-generation of recommendations based on citizens’ visions

Time horizon for ideas

40 to 50 years

Maximum time horizon 10 to 20 years

Participants

Heterogeneous group of citizens

Heterogeneous stakeholder and expert groups

Input information

Prompting material to inspire citizens thinking about future

Citizens’ Visions, Analysis report on key issues and needs

Expected outputs*

Visions (desirable futures) with focus on emotional knowledge

Recommendations on research and innovation policy based on visions with focus on cognitive knowledge

Addressees

A broad spectrum of actors including experts, stakeholders, citizens and policymakers in different fields.

Citizens and, after validation, policy makers and administration, also the research community

Application of results

New input on societal needs into research programmes and policy debates

Validation by citizens and policy debates, research agendas

* CIVISTI is designed to focus on different types of knowledge for decision-making in category b at different steps.

Tab. 1: Two different workshop types in the CIVISTI method. Source: own compilation

Introduction to the CIVISTI method

CIVISTI proceeds in three main steps. step one: generation of citizens’ visions; step two: generation of experts’ recommendations based on citizens’ visions; and step three: validation of recommendations by citizens. Step 1 involves a diverse range of background knowledge of citizens and is framed to provide target knowledge through their visions for desirable futures, which are then analysed by experts and stakeholders in step 2. This step is framed for using experts’ systemic and transformation knowledge in order to identify underlying societal expectations. Citizens’ visions aim to include expectations (hopes and concerns as emotional knowledge for decision-making) in a standard vision format – of about one page of text including a short description of a desirable future in about 40 to 50 years and dealing with concerns, hopes and reflections on possible positive and negative impacts of this future. These visions are the basis for experts’ and stakeholders’ cognitive analysis to decide on recommendations for research programme development, which are afterwards returned to the citizens for evaluation and prioritisation (step three of CIVISTI).

In Table 1, we compare the first two steps, which are as mentioned, mainly responsible for the co-generation of knowledge and are the focus of this contribution. The third step, which is less standardised than the first two, is a reflection of the expert evaluation by the citizens.

The CIVISTI method has been developed and applied during the last ten years in a number of EU and nationally funded projects in different scopes (see Table 2).

The CIVISTI method is based on a commitment to the inclusion of society and deliberation on targets for technology and innovation policy, in order to respect the diversity of opinions and perspectives, the willingness of politics, business and society to think out of the box for the transformation of socio-technical systems, moving towards a just society for present and future generations and fostering creativity for visioning. In the following sections, we show the role of different types of knowledge for creativity and out of the box thinking by citizens and generation of recommendations by experts.

Requirements for fostering creativity in CIVISTI

In this section, we show how creativity is fostered in CIVISTI through the three categories of knowledge mentioned above.

Fostering lay creativity through a balance of homogeneity and heterogeneity of groups and fair discussion

The actor-specific category of knowledge co-generation (see category a, introduction and background) implies a precise definition of the participant groups. There are different understandings of the role of lay participation for creative visioning in different foresight methods. One position emphasises the early inclusion of lay people’s knowledge and of groups such as women and children, who are usually excluded from decision making, in creative visioning (Masini 2006). By contrast, a primary focus on experts’ knowledge and secondary discussion with lay people is advised: “the Casual layered analysis, pioneered by Sohail Inayatullah, which provides a framework for structured discussions by lay people […] having in mind the existing ‘knowledge-power’ structure. He sees the role of the laymen as bringing the specialists’ knowledge down to earth and foreseeing its possible side-effects in everyday life” (Nikolova 2014, p. 4).

CIVISTI-based Projects*

Years

Scope

Scale

No. of Citizen Visioning Workshops

No. of citizens

No. of Visions created

No. of experts and stakeholders

CIVISTI

2008–2011

Research programme development for Horizon 2020

EU
8 countries

8

200

80

30

CIVISTI-AAL/Leben2050

2013–2014

Consulting for City development; autonomous ageing in future cities

Regional: City of Vienna

1

50

10

50

Future Foods 4 Men & Women

2014–2016

Research programme development for the Austrian Agency of health and food safety

National: Austria

5

90

50

30

CASI

2015–2017

Framework for assessment of social and technical innovations

EU
12 countries

12

226

50

500

* The EU-funded project CIMULACT (www.cimulact.eu) is based on the CIVISTI method, but its method differs considerably and it is therefore not considered here.

Tab. 2: Overview of a number of workshops and participants in different CIVISTI-based projects. Source: own compilation

CIVISTI follows the first strategy and starts with a fair and inspiring lay visioning process. The citizen panels have been homogeneous in terms of the national or local context and heterogeneous in terms of age, education, gender, occupation, etc. In this way, it is possible to use the advantages of a minimum level of common cultural background and at the same time to promote individual openness to cooperation, since the participants are interested in discussions on future and are not invited as representatives of a special group. Framing the process for a long-term perspective to look 40 to 50 years into the future, citizens are inspired to think out of the box and focus on target knowledge (Table 1) without any experts present. The diversity of citizens for a panel based on different experiences has a strong influence on the quality of discussion. Different authors report that “Participants in dialogue speak and listen with mutual authenticity and openness, seeking to understand and learn from each other’s experience and perspective without refuting the legitimacy of divergent views” (Abdel-Monem et al. 2010, p. 748). The evaluation report of the first citizens’ visioning workshops in CIVISTI in 2008 shows clearly that the majority of citizens had a very positive view of the fairness of the discussion and the creative environment for thinking out of the box. Later experiences confirmed this (Bedsted et al. 2017).

Fostering creativity through inspiring elements in lay-visioning

The design of the visioning process encourages citizens to start with emotional knowledge (Table 1) and develop visions in a step-by-step, moderated, inspiring process (Table 3). The visioning workshops are usually conducted in one or two days depending on the scope of the case study.

Action

Process with regard to fostering creativity in CIVISTI

1. Sharing knowledge on challenges and developments of the past

Different groups of 6 to 8 people start to share knowledge in an indirect way, prompting a sense of opportunity for change over a longer period; Fostering group affiliation by communicating personal stories

2. Visual inspiration

Facilitation of individual learning; Fostering group affiliation, futures thinking and creativity inspiring through pictures

3. Fast dreaming

Facilitation of generation of ideas and group communication to share ideas, using prompting questions

4. Clustering of ideas

Selection and reflection on thoughts; Finding differences and commonalities through citizens themselves

5. Market of ideas

Information and overview on possible ideas; Building a shared understanding of desirable futures in an exhibition of ideas of all groups

6. Imaginary time travel

Facilitation of knowledge co-generation; Providing calm space for individual creativity and processing through reading a short story in plenum

7. Deciding and merging of stories to create raw visions

Starting with the direct knowledge co-generation in small groups

8. Doing the first steps of visioning

Focus on a special part of knowledge and specify a story in small groups

9. Decision on raw visions and selection

Reflection phase; Providing a space for mutual learning between different groups

10. Final visioning and transcript, presentation

Reflection on consequences and improvement of knowledge in small groups and presentation in plenum

Tab. 3: design of steps and functions of visioning process regarding knowledge co-generation. Source: own compilation

Observation of the CIVISTI workshops has shown that citizens focus sometimes not only on emotional knowledge, but also on pragmatic, normative or cognitive knowledge for decision-making (category b). In such cases or also, when they focus on transformation knowledge instead of target knowledge (category c), it reduces the condition of thinking out of the box. One of the observed difficulties of the last step is that citizens reduce sometimes the creativity of their ideas at the end of the workshops, since they repeatedly fall back to thinking in terms of feasibility based on pragmatic and cognitive knowledge. A success factor for the facilitation of the visioning process is the promotion of creativity during the whole visioning process and reduction of focus on economic advantages or perceived feasibilities. As an example: citizens who generated a story with the wish for new multi-generational living concepts in the city in the Leben2050 project started to reduce the creativity of their visions and the focus on their emotional knowledge as soon as they discussed the costs of such processes for the whole city. Here, they were going to leave the borders of the step 1 and enter the step 2 of the process (Table 1). As citizens’ creativity was reduced by thinking about feasibility, facilitators suggested they could think about possible financial aid in 40 years and continue to work on specification of the new living models of the future. Citizens then discussed different new ideas in order to fulfil their daily needs and to be happy in the new living concepts. Here, the focus was on the emotional dimension (a sub-category of knowledge category b). Citizens were informed that experts and stakeholders were responsible for considering feasibility analysis for the next 20 years based on systemic and transformation knowledge and focus on cognitive knowledge (Table 1). Finally, experts could identify the need of social integration and the concern of social isolation as a core issue in different visions. They could define a number of feasible measures[2] such as design of real life meeting spaces in addition to ambient assisted technologies in new living concepts to “avoid social isolation of older adults”. The latter issue is based on normative dimension (category b), which should be avoided to reduce social divide.

After this short example of involvement of different knowledge categories within CIVISTI, the next section discusses the application of knowledge analysis for evaluating the quality of the process and results of knowledge co-generation.

Towards criteria for evaluating knowledge co-generation for CIVISTI-based case studies

For the evaluation of knowledge co-generation, one important factor is the separation of roles of actors. CIVISTI is designed for clear roles of citizens within the lay visioning process and also between citizens, stakeholders and experts. The separation or combination of actors’ roles in the visioning process influences lay-visioning. If, for example, experts and stakeholders are chosen intentionally or inadvertently during recruitment to join the visioning process (workshops), they influence the results with their expertise.

For strategic planning, it is of utmost importance to strengthen citizens’ roles as visionaries to elicit wishes and concerns for the future, while keeping their focus away from everyday systemic knowledge (e. g. today’s newspaper headlines). The contribution of citizens to transformation knowledge is separated from the visioning step to avoid feasibility thinking reduce their creativity. At the same time experts are asked not to develop their own visions while analysing and transforming citizens’ visions in the recommendations. Evaluating knowledge co-generation should closely observe if the role separation has been successful (see also ZSI 2011). Nevertheless, citizens should be able to provide their knowledge on feasibility to the third step of the co-generation process for the validation of recommendations (see introduction to the CIVISTI method).

CIVISTI emphasizes the early inclusion of lay people’s knowledge into target and priority setting for research and innovation by means of creative visioning.

In comparison to the citizens’ focus on emotional expectations in visions, the CIVISTI process is designed so that the stakeholders and experts analyse citizens’ visions mainly based on their cognitive and pragmatic considerations including practical guidelines and tools for decision-making at local, national and international levels, and normative frameworks including regulation and considering transformation knowledge. The aim of the facilitation concept in step 2 is thus that recommendations should be based on the cognitive knowledge of stakeholders and experts for the analysis of the emotional knowledge of citizens. Nevertheless, our observations show that experts might also leave the framework of step 2 and try to develop target knowledge (category c, Table 1) or focus on emotional knowledge instead of cognitive knowledge for decision-making (category b, Table 1). In such cases, there is a risk of reducing validity of the produced recommendations. Facilitators should therefore be prepared to deal with introduction of other types of knowledge to the discussion and at the same time emphasize the role of experts in step 2 for generation and selection of recommendations as mentioned in Table 1. It should also be mentioned that one of the criteria for validation of recommendations in step 3 by citizens is the quality of step 2, which is often called as the loyalty of experts and stakeholders to citizens’ visions. In sum the compliance of a CIVISTI case study with the standard rules of the method (Table 1) depends strongly on the special roles of actors of step 1 and 2 and special focus on different types of knowledge at each step. This clear structure supports also addressees of recommendations (Table 1) to identify future citizens’ needs and experts and stakeholders’ recommendations to improve research programmes based on these needs.

Conclusion

Assumptions about participation are often rooted in 20th century, expert oriented culture of thinking. We observe a need for more spaces where individuals can enter into profound discussions about concerns and hopes for the future of the environment, economic development and a just society.

New creative participatory foresight processes in TA could be milestones for a transformation towards the institutionalisation of future-oriented knowledge co-generation able to deal with grand challenges. Fostering creativity and conducting the visioning process in a fair democratic environment can serve as a valid inclusion of laypeople in agenda setting, and therefore increase legitimacy and accountability of research and innovation agendas. For evaluation of participatory foresight processes in TA there is a need for both standard criteria for evaluation of public participation such as fairness, transparency and efficiency as well as a clear understanding of the knowledge co-generation process.

The suggested focus on the role of different categories of knowledge at different steps of the participatory foresight processes supports structural transparency and should be discussed during the design and implementation of the processes as well as the analysis of results. It should help to develop more precise facilitation concepts according to a higher level of awareness on required contributions at each step and it could support better insight to the quality of results of the participatory foresight in TA.

As an example, in CIVISTI we should be sure, if citizens from different backgrounds could generate new knowledge based on their own hopes, concerns and everyday experiences regarding a desirable future, or if stakeholders and experts have influenced the visioning process through presentations, moderation, etc. Therefore, the evaluation approaches should consider, whether actor-specific roles have been explicitly defined and taken into account for knowledge co-generation.

Further questions for CIVISTI are whether the focus of citizens is on target knowledge, as planned, and whether the stakeholder and experts’ workshops focus on systemic and transformation knowledge according to the design of the process. Therefore, in future, we will distinguish explicitly between different dimensions of knowledge and knowledge hierarchies for the design of facilitation concepts and evaluation of knowledge co-generation in CIVISTI-based case studies.

A closer look into knowledge co-generation is also useful for other participatory foresight methods in TA. In this way, we hope to be able to identify weak points of the knowledge co-generation from a new perspective and also to achieve improvements in development of participatory foresight methods in TA.

Footnotes

[1] “A cognitive dimension, which reflects different actors’ perspectives on scientific and technological issues […], a normative dimension, reflecting the plurality of (possibly conflicting) norms and values […], a pragmatic dimension, reflecting the unequal distribution of institution- alised or informal influence on decision-making processes” (Belluci et al. 2002, p. 18).

[2] Transformation knowledge for long-term planning (category c), pragmatic dimension (category b).

References

Abdel-Monem, Tarik; Bingham, Shereen; Marincic, Jamie; Tomkins, Alan (2010): Deliberation and diversity, perceptions of small group discussions by race and ethnicity. In: Small Group. 41 (December), pp. 746–776.

Bedsted, Bjørn et al. (2017): European citizens’ visions for a sustainable EU future, Research priorities and policy advice. CASI project Deliverable 3.3. Available online at http://www.casi2020.eu/library/deliverables/, last accessed on 23. 01. 2018.

Bellucci, Sergio et al. (2002): Theoretical perspectives In: Simon Jossand Sergio Belluci (eds.): Participatory technology assessment. European perspectives. Gateshead, UK: Athenaeum Press.

Brătianu, Constantin; Orzea, Ivona (2014): Emotional knowledge. The hidden part of the knowledge iceberg. In: Management Dynamics in the Knowledge Economy 2 (1), pp. 41–56.

Dubielzig, Frank; Schaltegger, Stefan (2004): Methoden transdisziplinärer Forschung und Lehre. Lüneburg: Centre for Sustainability Management, Universität Lüneburg. Available online at http://www2.leuphana.de/umanagement/csm/content/nama/downloads/download_publikationen/49-8downloadversion.pdf, last accessed on 23. 01. 2018.

Gudowsky, Niklas; Peissl, Walter; Sotoudeh, Mahshid; Bechtold, Ulrike (2012): Forward-looking activities, incorporating citizens’ visions. In: Poiesis & Praxis 9, pp. 101–123. DOI: 10.1007/s10202-012-0121-6.

Gudowsky, Niklas; Peissl, Walter (2016): Human centred science and technology. Transdisciplinary foresight and co-creation as tools for active needs-based innovation governance. In: European Journal of Futures Research 4 (8). DOI 10.1007/s40309-016-0090-4.

Gudowsky, Niklas; Sotoudeh, Mahshid (2017): Into blue skies. Transdisciplinary foresight and co-creation as socially robust tools for visioneering socio-technical change. In: NanoEthics 11 (1), pp. 93–106.

Gudowsky, Niklas; Sotoudeh, Mahshid; Capari, Leo; Wilfing, Harald (2017): Transdisciplinary forward-looking agenda setting for age-friendly, human centred cities. In: Futures 90 (6), pp. 16–30.

Jacobi, Anders; Klüver, Lars; Rask, Mikko (2010): Relevant research in a knowledge democracy. Citizens’ participation in defining research agendas for Europe. In: Roeland in ’t Veld (ed.): Knowledge Democracy. Heidelberg: Springer, pp. 87–98.

Joss, Simon; Bellucci, Sergio (eds.) (2002): Participatory technology assessment. European perspectives. Gateshead, UK: Athenaeum Press.

Kuznetsov, Nikita; Baksanskii, Oleg; Zholkov, Sergej (2012): Sources and foundation of pragmatic knowledge. In: Journal of Communications Technology and Electronics 57(8), pp. 868–881.

Masini, Eleonora (2006): Rethinking futures studies. Futures 38, pp. 1158–1168.

Nikolova, Blagovesta (2014): The rise and promise of participatory foresight. In: European Journal of Futures Research 2: 33, 9 pp. https://doi.org/10.1007/s40309-013-0033-2.

Nowotny, Helga (2003): Dilemma of expertise. Democratising expertise and socially robust knowledge. In: Science and Public Policy 30, pp. 151–156.

Ostrom, Elinor; Parks, Roger; Whitaker, Gordons (1978): The public service production process. A framework for analyzing police services. In: Policy Studies Journal 7 (1), pp. 381–389.

Pohl, Christian; Hirsch Hadorn, Getrude (2006): Gestaltungsprinzipien für die transdisziplinäre Forschung. München: oekom.

Polk, Merrit; Knutsson, Per (2008): Participation, value rationality and mutual learning in transdisciplinary knowledge production for sustainable development. In: Environmental Education Research 14 (6), pp. 643–653.

Ramírez, María-Soledad; García-Peñalvo, Francisco-José (2018): Co-creation and open innovation. Systematic literature review. Comunicar 26 (54), pp. 9–18.

Regeer, Barbara; Bunders, Joske (2009): Knowledge co-creation: Interaction between science and society. A transdisciplinary approach to complex societal issues. Den Haag: RMNO Publications.

Renn, Ortwin (2009): Integriertes Risikomanagement als Beitrag zu einer nachhaltigen Entwicklung. In: Reinhold Popp and Elmar Schüll (eds.): Zukunftsforschung und Zukunftsgestaltung. Beiträge aus Wissenschaft und Praxis. Berlin: Springer, pp. 553–568.

Rower, Gene; Frewer Lynn (2005): A typology of public engagement mechanisms. In: Science, Technology and Human Values, 30 (2), pp. 251–257.

Sotoudeh, Mahshid; Gudowsky, Niklas; Capari, Leo (2014): Wünsche an die Zukunft. Zukunftsvisionen Jugendlicher über eine nachhaltige Entwicklung. In: Umweltdachverband GmbH (ed.): Krisen- und Transformationsszenarios. Frühkindpädagogik, Resilienz und Weltaktionsprogramm (Bildung für nachhaltige Entwicklung, Jahrbuch 2014). Wien: Forum Umweltbildung, pp. 124–130.

van Veen, Saskia; Bunders, Joske; Regeer, Barbara (2013): Mutual learning for knowledge co-creation about disability inclusive development. Experiences with a community of practice. In: Knowledge Management for Development Journal 9 (2), pp. 105–124.

Voorberg, William; Bekkers, Viktor; Tummers, Lars (2015): A systematic review of co-creation and co-production. Embarking on the social innovation journey. In: Public Management Review 17 (9), pp. 1333–1357.

Research Data

Websites of CIVISTI-based Projects: CIVISTI: www.civisti.org; CIVISTI-Ambient Assisted Living (Leben 2050): www.oeaw.ac.at/ita/en/projects/civisti-aal/overview; Future Foods 4 Men & Women: www.futurefoods.ages.at/home; CASI: www.casi2020.eu.

Brandstetter, Regina; Gajdusek, Martin; Kesselring; Alexander; Schuch, Klaus (2011): Evaluation of the FP7 project CIVISTI. Final report. Vienna: Centre for Social Innovation. Online available at http://www.civisti.org/files/images/D_6_3_Evaluation_report.pdf, last accessed on 23. 1. 2018.

Authors

Prof. Dr. Mahshid Sotoudeh

is a TA researcher with a background in chemical and process engineering. Her research areas are technology and sustainability, participatory methods and foresight. She is associate professor in the field of “technology assessment and sustainability” at Technical University Graz.

Dr. Niklas Gudowsky

is a TA researcher with a PhD in Biology. He focuses on participatory foresight for agenda setting in science and technology and is also editor of the ITA dossiers, a publication series which summarises core results of current research with a focus on options for action in politics and society.