Quality Assurance by Extended Peer Review: Tools to Inform Debates, Dialogues & Deliberations

In this paper the challenges of trans-disciplinary practice and quality assurance by extended peer review, in terms of knowl-edge co-production, mediation and repre-sentation, are addressed. Spaces for the articulation of plural narratives are ex-plored, including the opportunity to deploy new information technologies. The TIDDD (tools to inform debates, dialogues & delib-erations) is introduced in the context of the GOUVERNe project. 1 The Trans-disciplinary Challenge


The Trans-disciplinary Challenge
It is now recognised that fields of knowledge and scientific practice are, in many cases, no longer usefully divided into isolated compartments.This awareness leads to attempts to create bridges among several disciplines, and the emergence of inter-disciplinary and multidisciplinary studies and projects.Already in the 1960's multi-disciplinary approaches were seen as an essential way to tackle practical problems, providing an impulse for the further development of systems thinking, integrated methodologies and operational research.The late 1970's academic curricula everywhere reflected the recognition that societal issues have to be approached, framed, resolved and justified from a multiplicity of perspectives, some even recognising a trans-disciplinary evolution (Nicolescu 1999).This is clear in addressing problems of sustainability and, in general, environmental governance.
There are several definitions of transdisciplinarity but it is generally described as a specific form of inter-disciplinarity in which boundaries between and beyond disciplines are transcended and knowledge and perspectives from different scientific disciplines as well as non-scientific sources are integrated (Flinterman et al. 2001;Klein et al. 2001).
The above definition stresses the importance of integration of different scientific dis-ciplines and non-scientific sources (and types) of knowledge.The latter is the characteristic feature that distinguishes trans-disciplinarity from other multi-disciplinary approaches.The concept originated from the increasing demand for relevance, legitimacy and applicability (which are aspects of quality) of academic research to the challenges of societal problematiques in a policy context.Jasanoff (1996) has argued that scientific knowledge is not independent of political context but co-produced by scientists and the society within which they are embedded.The contextualisation of knowledge production, as policy relevant scientific knowledge requires a trans-disciplinary approach, both in terms of the integration of types of knowledge and of the mediation among sources of knowledge.Trans-disciplinarity is unavoidable in knowledge production, it being implicit in the context in which knowledge co-production occurs or explicit in the integration of different types and sources of knowledge.
Each discipline has developed in an established conceptual and methodological framework, with its own scales, language, narratives, knowledge representation, knowledge mediation and communication.Also scientisation has lead to the scientific internalisation of many societal issues, especially in environmental and health domains.This has occurred mainly through the creation of masses of quantitative knowledge and argumentations which, in many cases, do not help and can even worsen controversies (Sarewitz 2004), confusing framings and justifications when (urgent) action is required.
In this paper, we concentrate on the development of a tool (TIDDD or tools to inform debates, dialogues & deliberations) in order to operationalise the concept of quality assurance through an extended peer community, as a response to the need to extend the assessment of relevant knowledge to those who contribute to its co-production, outside the boundaries of disciplinary science.

Quality and Extended Peer Review
The assessment of the quality of the knowledge inputs to policy issues are in many ways different from those of research science, professional practice or industrial development (Funtowicz 2001).Each of those has its established means for quality-assurance of the products of the work, be they peer review, professional associations, or the market.However, for new controversial problems, the maintenance of quality depends on open dialogue between all those affected.This we call an "extended peer community", consisting not merely of persons with some form or other of institutional accreditation, but rather of all those with a desire to participate in the resolution of the issue (Funtowicz, Ravetz 1990).Since this context of science is one involving policy, we might see this extension of peer communities as analogous to earlier extensions of the franchise in other fields, such as women's suffrage and trade union rights.
Extended peer communities are already being created, in increasing numbers, either when the authorities cannot see a way forward, or when they know that without a broad base of consensus, no policy can succeed.They are called "citizens juries", "focus groups", "consensus conferences", or any one of a great variety of other names; and their forms and powers are correspondingly varied.But they all have one important element in common: they assess the quality of policy proposals, including a scientific element, on the basis of the science they master combined with their knowledge of the ways of the world.The contribution of relevant social actors in this case is not merely a matter of broader democratic participation and their verdicts all have some degree of moral force and hence political influence.
These extended peer communities will not necessarily be passive recipients of the materials provided by experts.They will also possess, or create, their own "extended facts".These may include craft wisdom and community knowledge of places and their histories, as well as anecdotal evidence, neighbourhood surveys, investigative journalism and leaked documents.Such extended peer communities have achieved enormous new scope and power through the Internet.Activists scattered among large cities or rainforests can engage in mutual education and coordinated activity, providing themselves with the means of engagement with global vested interests on less unequal terms than previously.
Along with the regulatory, evaluative function of extended peer communities, another, even more intimately involved in the policy process, is springing up.Particularly at the local level, the discovery is being made, again and again, that people not only care about their own environment and health but can also become quite ingenious and creative in finding practical, mixed social and technological means for their improvement.In many cases, local people can imagine solutions and reformulate problems in ways that the accredited experts, with the best will in the world, do not find normal.This is most important in the phases of policyformation, and also in the implementation and monitoring phases.Thus, in addition to extending the traditional processes of quality assessment, participants can enhance the quality of the problem solving processes themselves.
As stated earlier, trans-disciplinary practise arose as a response to the increasing complexity of scientific knowledge production, and the need to re-establish an active dialogue among a plurality of disciplines and forms of knowledge (Nicolescu 1999).This requirement now extends beyond the inter-operability of methods and techniques coming from different scientific disciplines; it is in fact a quest for quality, not (just) excellence in scientific terms, but robustness also in societal terms (Gibbons 1999).The aim of knowledge quality assurance by extended peer review is precisely to open processes and products of policy relevant science to those who can legitimately verify its relevance, fitness for purpose and applicability in societal contexts, contributing with "extended insights and knowledge".
Trans-disciplinary practice and extended peer review face common challenges such as, for example, resistances and closure of institutional or established practice in research and policy, different conceptual and operational framings, knowledge representations and mediation.The remaining of this paper will address the issue of knowledge representation and mediation.

Knowledge Representation & Mediation
In trans-disciplinarity literature, the issue of knowledge representation and communication is recurrent due to a perceived need to communicate more complex and dynamic insights, exploring the use of metaphors, patterns and analogy (see e.g.Judge 1995).Knowledge representation and mediation become an issue when different sources and types of knowledge have to be "integrated" or "fit in a framework of analysis" or have to be articulated in a same decision space.It often happens that the need to deal with a diversity of knowledges originates from those who are already used to a certain type of framing and the deployment of specific tools of assessment.
Traditionally, integration meant reductionism and the conversion of different languages into one single, mainly quantitative language (e.g.Cost Benefit Analysis or other monocriterion evaluation techniques, such as multiattribute theory).This tendency has persisted, despite the pitfalls of knowledge loss, poor scoping and increased controversy.The recognition of multiple perspectives has encouraged the use of frameworks trying to acknowledge and to operationalise a diversity of knowledge representations.Among such frameworks are multicriteria evaluation (see, for instance, Munda 1995), integrated assessment modelling (see, for instance, Alcamo et al. 1994), multi-scale integration (Giampietro 2003).These attempts arise from the need to make comparisons, seek for trade-offs or even become Alephs (the place from where all dimensions could be seen at the same time, according to the poet J. L. Borges (in El Aleph written 1949 -see for instance Borges 2001) regarding alternative courses of action, policy making options and foresight.
In trans-disciplinary practice and extended peer review it is often the case that the framework in which knowledge integration and assessment occurs is that of research, characterised by concepts and tools that determine in a sense the ways in which knowledge has to be represented in order to be shared.Among others Giampietro (2003) talks about different narratives depending on who initiates the process, with which purpose, perspectives and values.Hence, the main challenges posed to transdisciplinary research, extended peer review and other attempts to integrate different knowledge sources and types are, on one hand, the creation of spaces for knowledge representation and mediation and, on the other hand, the creation of spaces for knowledge co-production.
We argue that such methodologies should ultimately provide spaces to make sense of a variety of bits of knowledge, recognising the legitimacy of, not only different types of knowledge, but also different ways of articulating them.It is not surprising then that transdisciplinary literature points to the use of metaphors, patterns, multi-media visualisation, isomorphism, analogies, and to methods that acknowledge diversity and complexity.
A promising development of this sort of knowledge conviviality is the creation of contexts of co-production of knowledge, entailing different types of knowledge organisation for non-scientific contexts, experimented in sustainability issues (see for instance, Guimarães Pereira, O'Connor 1999).We advocate that, as in extended quality assurance, trans-disciplinary practise requires a new type of skill, enabling the creation of multiple interfaces between scientific and non-scientific knowledges.

Trans-disciplinarity at Work: Case Study on Groundwater Resources
The project GOUVERNe 2 aimed at the development and pilot implementation of a userbased scientifically validated process and informatics product for the improved governance of groundwater resources.The Joint Research Centre activities in this project dealt mainly with the organisation of the available knowledge about two groundwater resources case studies in Europe (see Guimarães Pereira et al. 2003a andGuimarães Pereira et al. 2003b).
The methodology deployed was based on the concept of quality assurance by extended peer review, as a normative procedure to construct the knowledge base upon which a debate about water governance options could start in both case studies.What we called the "GOU-VERNe process" was strongly based on transdisciplinary principles, combining hybrid methodologies, integrating social research methods with evaluation tools, such as multicriteria evaluation.

The GOUVERNe Process
Knowledge scrutiny in the GOUVERNe process was strongly based on social research.That was the means to ensure that knowledge other than scientific-technical was available in the knowledge base to debate on possible futures for groundwater resources and the associated river basins of the two case studies (in France and Greece).
The involvement of relevant social actors was done from the very first framing step which ensured that, early in the process, their perspective, concerns and ways of representing the issues were accounted for.The research framing acknowledged and shared by the relevant community helps to avoid the so called Type III error, of addressing the wrong problem, and enhances the scoping phase (i.e.focuses the work of collecting relevant information).The extended involvement also means that the issues addressed are shared and are dealt with at the appropriate depth.
Clearly, if the experts involved in the process are the only "digesters" of the available knowledge (even if the process is inclusionary), their research framing and representation will be paramount.This is why the quality check by the relevant community throughout the whole process is essential for compliance and effectiveness, and why the process of creating socially robust knowledge (Gibbons 1999) is a continuous inclusionary process of those concerned.In GOUVERNe engagement of relevant social actors was done at several steps of the process.
What emerged from the processes of knowledge scrutiny is that activities and options explored together by those concerned, had great advantages in terms of enhancing the final process of dialogue, compared with those activities structured solely by "experts": what becomes available as knowledge base to support the ongoing dialogues is perceived as a coproduced result and issues are then more easily appropriated by all those engaged.
One of the main research issues of this process was how to articulate different values and perspectives, as well as different representations of knowledge which may be presented through alternative narratives (language, framing, scales of measurement, numerical models, etc).GOUVERNe is about knowledge integration, while trying to keep diversity which in the interpretation of the researchers was the creation of a space: where different types of knowledge articulated in different sets of semantic rules, with different codes, different scales of evaluation, etc. could be represented through several formats implying various degrees of specialisation; where no a priori "integrative methodology" was applied as the means of sharing knowledge, the integration being made through dialogue and interactions.
This entails the effort to produce a sort of "knowledge platform" that is accessible to all those involved and promotes conviviality of different knowledges, including tools that help with the process evaluation, capturing plurality and diversity and avoiding the pitfall of reducing them to something plausible but meaningless.This was explored through the use of Information and Communication Technologies (ICT) and in particular multi-media knowledge representation.

Building Spaces for Conviviality: The TIDDD Concept
A major development within the GOUVERNe process was the realisation, the design and prototype implementation of a new concept tool: TIDDD -Tools to Inform Debates, Dialogues & Deliberations, deploying new ICT.The main characteristics of this tool can be defined as "tools that inform and mediate processes of debate, dialogue or deliberation which involve social actors of a governance, policy or decision process".Mediation of knowledge in this case entails organisation, communication and exchange of a plurality of sources and types of knowledge (Guimarães Pereira et al. 2003b).In the case of GOUVERNe, there was a great deal of disciplinary knowledge, such as climate, geological and hydrological, as well as socioeconomical, regulatory, etc. Scenario drivers to debate about future options were devised together with the social actors.Hence, as some modelling tools were used to characterise possible futures, there had to be some work on "translating" that information in order to use it as input for the models.TIDDD's aim is the creation of convivial contexts of exploration, and "discovery", where representations of knowledge come from different actors in the form of consistent narratives, aided by a multiplicity of supporting materials, namely multi-media formats, metaphors, etc.In TIDDD some pieces of information were represented through different media in order to reach different people involved.TIDDD can integrate other sources and types of knowledge that may emerge during the process, which is done through the available multi-criteria evaluation tool.Quality assurance through extended peer review of TIDDD contents and design is one of the basic principles of this tool, since its main aim is to provide socially robust knowledge in contexts of societal debates, and even scientific controversy.This is achieved through upstream engagement of the relevant community in the implementation of the knowledge base available in TIDDD, where the social actors check all developments and ensure that contents and design are suitable to start the debate on groundwater resources futures.

Final reflection
TIDDD-like tools are interfaces of mediation between policy spheres and other sectors of the society.This mediation is done with the help of experts, but what comes out of the GOU-VERNe process is that a new class of expert is emerging, experts in creating contexts for coproduction of knowledge, experts in mediation of different types of knowledge, perspectives and values, and eventually experts in making scattered non-organised pieces of relevant knowledge intelligible to the organised and sometimes poorly flexible institutions: in a sense trans-disciplinary experts.
Trans-disciplinarity practice and extended quality assurance processes are about conviviality of different knowledges.It is hoped that tools like TIDDD can help to create the spaces where co-production and integration take place.The GOUVERNe TIDDD are in fact a transdisciplinary platform.
Notes 1) The views expressed in this article are those of the authors and do not represent necessarily those of the European Commission.2) Project no.EVK1-1999-00032: a Shared Cost Action financed by DG RTD, under FP 5. GOUVERNe stands for Guidelines for the Organisation, Use and Validation of information systems for Evaluating aquifer Resources and Needs.Online available at: http://neptune.c3ed.uvsq.fr/gouverne/ and http://alba.jrc.it/gouverne.