Page locked

This page is currently locked for editing by another user. You have to wait until this user finishes editing or the lock expires.
Currently locked by: 111.225.149.4
Lock expires at: 2024-05-08 20:07 (4 min)
This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== Marine CoLAB Reflection =====

Notes from the Marine CoLAB evaluation and reflection meeting at CGF in London, on the 22nd of January 2016. 

Present: Louisa Hooper, Margaret Bolton, Esther Goodwin Brown, Vali Lalioti, Maja Kuzmanovic and Nik Gaffney


==== CGF aims ====

Marine environment is complex, multi-stakeholder, views on its values diverge. Science and Economics dominates decision making, and CGF perceives the need to make issues 'human'. The long term aim of Marine CoLAB is to improve conservation and management of oceans for human wellbeing, as well as to make connections and build relationships that would increase capacity of the sector. CGF expects that this can be accomplished through:
  * Effective flows of knowledge (to deepen understanding of the role of the ocean on human wellbeing, as well as to better understand priorities for action and inspire others to act in new ways)
  * Improved collaborations and shared learning across borders, sectors and disciplines (to improve sharing and managing ocean resources)
  * Deployment of new skills & resources (increased effectiveness, influence, capacity, innovation, audience engagement)
 
Marine CoLAB was established to use a //lab// approach to work intensively with a small group of change-makers with skills and ambitions to increase impact of the sector. 

Objectives:
  * identify changemaking NGOs; participants, advisers, influencers, strategic partners, funders.
   * establish a cross sector platform, which would include:
    * programme of workshops between  2014 & 2016 (to build capacity of NGOs to deliver game changing initiatives)
    * portfolio of projects; new, existing, joint, synergies with GOI (to foster collaboration and innovation and be enabled by flexible funding and evaluation schemes)
    * an evolving, learning community
  * design an evaluation and dissemination plan
 

==== FoAM's Lab Approach ==== 

To meet CGF's aims FoAM proposed a [[/start|programme of workshops]] designed to enable a learning community and foster a development of:
   * a lab/platform/network for Valuing the oceans 
   * a portfolio of projects/experiments/activities/

The [[/start|programme]] was developed in two phases: a scoping phase and a transitional phase towards a self-guided CoLAB.

=== Scoping phase ===

[January - August 2015]

==Workshop 1:  Questions, vision, co-creation [January 2015]==
  * Before the workshop collect a set of 'key questions' and select one core question to guide the group through the session
  * Begin the workshop from personal experiences related to the core question ["worldchanging experiences]
  * Mapping what is know and what is unknown about valuing the oceans in the group
  * Uncovering emerging trends important for the CoLAB [critical uncertainties: climate change and growth economy]
  * Designing four alternative scenarios and answers to the core question. All focused on connections, gaps, values and innovation

==Workshop 2: Engagement and discovery based strategy [March 2015]==
  * Begin with what each individual brings to the CoLAB [skills, personalities, strengths, etc.] 
  * Look at current challenges the individuals and their organisations are facing, to uncover patterns in the sector
  * Form hypothesis based on existing challenges and form groups to begin exploring them
  * Design iterative experiments to test the hypotheses and learn about each other and the LAB approach in practice, through short [3 months] collaborations 

==Workshop 3: Experimentation, connection to work of organisations [May 2015]==
  * Identify emerging themes [from current work and Marine CoLAB experiments]: systems change, changing perceptions, public engagement
  * Continue developing existing experiments, add new ones that specifically address emerging themes
  * Test experimental approach in a 'pre-enactment' [an experiential futures technique], a role-playing exercise with 'antagonistic stakeholders'
  * Complete design of experiments and create implementation plan

==Workshop 4: Reflection, Looking back and looking forward [July 2015]==
  * Clarify learning from [results and process of] experiments
  * Introduce collaborative processes for enabling scaling up experiments using an iterative approach [backcasting, outcome pathways, six ways of thinking [six thinking hats]
  * Evaluate the experiments and Marine CoLAB scoping phase  [using the adaptive action cycle technique]

=== Transitional phase ===

[September 2015 - March 2016]

==Workshop 5: Strengthen direction and shared values; community building in a multi-day workshop on location in Portugal [September 2015]==
  * Collaborative process design
  * Connection to Marine NGO sector in Portugal
  * Field-work [proposed: learning journeys]
  * Important: mix of formal sessions, field trips to the sea and different forms of socialising [joint meals, long bus ride, site visits, shared experiences [exhibition, Oceanario, lighthouse...]
  * Begin co-creation of Marine CoLAB as a platform [melting pot, Marine Innovation Exchange, CoLAB...]
  * Mapping of MarCoLAB Incubator including existing projects by participants, as well as Marine CoLAB experiments, such as the SUPB Free London. 
  * Implementation plan for the proposal to the OAK Foundation.

==Workshop 6: Marine CoLAB past, present and future [November 2015]==
  * Evaluate of Marine CoLAB's first year [what worked, what needs work, what can we improve]
  * Clarify mission and vision of Marine CoLAB
  * Dig into the values connecting people and oceans [and the values based approach]
  * Map existing and emerging MarCoLAB incubator projects, with in-depth discussions of SUPB Free London and Game On

==Workshop 7: Marine CoLAB: who are we and where are we going? [January 2016]==
  * Values
  * Mission
  * Lab strategy (action learning cycle)
  * Incubator of experiments [including an update on  SUPB Free London]


== Workshop 8: Marine CoLAB action plan [February 2016]==
  * Plan for 2016 (activities and envisaged outcomes)
  * Evaluation framework for the activities in 2016
  * Action learning cycle 
  * Experiment incubator and project consulting
  * Towards a Marine CoLAB operational model


== Workshop 9: long term operational model[March 2016] ==
  * Preparing a business and operational plan
  * Handover facilitation to lab participants


==== Evaluation ====

=== Year One ===

==Strengths==
  * group -> team
  * adaptive collaboration w/ funders & participants
  * multi-day residential trips
  * connecting people
  * drive to embrace big ambitions
  * not just doing (a lab), but being (a lab, a community)

==Challenges==
  * moving from big projects to agile experiments
  * over-commitment and lack of time
  * (divergent) values (based approach)

==Learning==
  * Hypothesis: playful co-creation brings people together (proven)
  * Hypothesis: multi-day on site workshops contribute to community building (proven)
  * Hypothesis: systems change  experiments (lab approach) lead to increased innovation (needs work)
    * maybe it would help if the 'lab approach' itself would be tested as a 'toy system': introduce model by making, testing rather than just talking about it
    * this is a longer process than expected, with more resistance in the beginning.
    * it helps to have an environment in which it is safe to experiment, but this takes a lot of time to create 
    * the participants had difficulty to design experiments in iterative ways
      * possibly due to experts preferring to avoid getting out of their comfort zone, especially when they don’t know each other
  * Hypothesis: Ambitious projects can be gradually reduced in risk through iterative, experimental, phased approach. (the hypothesis works when applied, but it has proven difficult for the participants to design experiments rather than projects)
    * learn form failure
    * never have to go back to scratch
    * needs appropriate documentation and evaluation
  * Hypothesis: 2nd cycle will transition into self sufficiency 
    * facilitators should become redundant
    * group takes more ownership of the process to become more self sufficient
  * Hypothesis: there needs to be continuous evaluation and adaptation (proven)
  * Hypothesis: lab approach can get to an operational plan quicker than using other methods (to be tested and compared with other initiatives)
  * Hypothesis: being able to take lab learning back into organisations (to be evaluated more formally; it appears to be happening from the testimonials and informal conversations)


=== Evaluation Methods ===

Methods that CGF and/or FoAM have experience with:
  * In general, trying to find ways of using logic and planning tools 'in reverse' or parallel as evaluation tools
  * Direct feedback; listening circle, step-in step-back, Japanese post-its, debriefing, reflecting in a group
  * Written reports: Evaluation templates, qualitative evaluation
  * Interviews (sponsors, participants): impact conversations structured interviews
  * Before and after: goal resolution, identifying indicators of progress an measuring them, asking before and after questions
  * Appreciative Inquiry: AI focuses on positive attributions e.g. what is working well, by drawing on case examples which are examined for key themes and emerging patterns.
  * Adaptive Action Cycle (what, so what, now what)
    * Ethnographic approaches: participant-observer, visual (documentaries), interviews, stories, collecting and evaluating participant reports
  * Theory of change (both as design/planning and evaluation method) aka ToC. ToC can be used as  learning object, providing a specific focus for feedback, evaluation and discussion around outcomes and assumptions. However, it has been critiqued as too static. 
  * Outcome mapping
  * 'Most significant change’
  * Logic analysis; logic models of projected impact, an adaptive ToC
  * Participatory impact pathway analysis (PIPA); problem tree, visioning, network perspective, outcomes in a logic model
  * Community based participatory models with stakeholders / groups / close others. interviews to determine impact / outcomes / experience (e.g. Community Voice)
  * Contribution analysis  - collective & individual contribution in networked governance
  * System mapping
  * FSG's work on evaluating collective impact initiatives; most significant change, appreciative inquiry, mapping
  * After action reviews / 'Post-mortem' for projects; objectives,process,results,people: what worked? what didn't? what did we learn? A structured review on what happened, why and how it can be done better e.g. for use in interviews, debriefs and workshops. 
  * 'Counterfactuals'
  * User / developer research (interviews, observation, (sensor) data i.e correlation of subjective and objective reports)
  * Evaluating unknowns (the black animals ( swan, elephant, jellyfish))
  * The 'ethanol' method


Challenge: finding appropriate ways of evaluating dynamic aspects of an initiative; the dynamics has to be present in the evaluation model as well.
  * circular, non-linear and complex mapping (e.g. MLP (multi layer perspective - Forum for the future))
  * mapping dynamics rather than static aspects subject to change.

References:
  * CGF's {{:marine_colab:glossary_of_terms.pdf|Glossary of Terms}}
  * [[http://www.fsg.org/publications/evaluating-complexity|Evaluating Complexity]]: Propositions for Improving Practice by Hallie Preskill, Srikanth "Srik" Gopal, Katelyn Mack, Joelle Cook
  * [[http://www.fsg.org/publications/guide-evaluating-collective-impact|Guide to Evaluating Collective Impact]] by Hallie Preskill, Marcie Parkhurst, Jennifer Splansky Juster