Posts

Evaluating the Long-Term Results of Environmental Education Curriculum in Traditional and Non-Traditional Educational Settings

There are many lesson plans and program activities available to teach students and participants about environmental issue analysis. But how often do educators go back and evaluate whether learners have processed the results, formed their own opinions, and potentially experienced a behavior change? The environmental education field would benefit greatly from the documentation of the evaluative results from these lessons and program activities, using shared concepts and language.

Most traditional and non-traditional educators who work to communicate environmental concepts to their community scour the internet or look to colleagues for ideas when teaching a new concept. A quick internet search on issue-based lesson plans and activities returns more entries than can be processed in a short period of time. Sometimes these lessons appear in blog posts like this describing their immediate application results. Often the discussions about learning outcomes are anecdotal based upon the educator’s experience with their participants during the activity itself.

Many environmental issue-based lessons are written for K-12 teachers or higher education faculty but there are other professionals who engage learners in non-formal learning about environmental or natural resource concepts. The language used to describe formal lessons or programs contains jargon specific to formal or non-formal learning environments but the methodology for evaluating learning outcomes is the same. Non-formal and non-traditional educators more often evaluate their programs, not just for participant learning achievements but for value-added and behavior change results due to expectations from funding organizations. Can all educators can benefit from measuring long-term results of our applied lessons and activities?

Following the Guidelines for Excellence EE Materials by NAAEE, foundational environmental education materials are:

  • Fair and accurate, considering a diversity of opinion and background experience
  • Conceptually deep, considering awareness, feelings, values, and attitudes
  • Hands-on activities for people of all ages designed to build life-long skills
  • Action oriented, a call to action via civic responsibility
  • Instructionally sound, using learner-centered concepts and include assessment
  • Useable and replicable by others for similar results

These concepts are broadly applicable to both formal/non-formal and non-traditional educators and should be considered when developing any content designed to teach issue-based, controversial environmental topics, especially when seeking behavior change.

Many models, including NAAEE’s Guidelines, recommend remaining unbiased, refraining from inserting emotive language, and using science-based concepts and data to communicate effectively. No matter the age of your students or participants creating a space that is safe where people can freely share their thoughts and values without judgement is important.

Discussion should be designed to:

  1. find common ground
  2. focus on facts and data supporting all viewpoints
  3. engage participants using hands-on and action-oriented activities
  4. evaluate and revisit the big picture.

The last step is critical to checking for understanding. If time has elapsed since the lesson delivery and the evaluation is specific to the issue-based engagement, educators can often determine if the activity had lasting effect.

Research-based literature describes methodologies for teaching controversial environmental topics, including those that have a strong human-centric influence and involve an understanding of how humans are part of the issue and potential solution.  These are difficult to teach because more often than not students of all ages are influenced by many external factors such as family values, morals, economics, other complicated ethos, or nihilism. Examples of controversial topics include:

  • climate change
  • animal production practices
  • nutrition
  • forest management
  • pesticide use
  • water quality

Despite nuance, presenting content grounded in scientific fact and identification of all possible perceived values can lead to sound decision making.

Formal educators can benefit from thinking beyond the curriculum and 5E lessons being taught during the current academic year with the current students. Developing a programmatic model for evaluation beyond testing current students for achievement of learning outcomes involves thinking about the lasting benefits both to current students as well as the application of curriculum in the future. Documenting these results and sharing them are excellent strategies for moving the field of environmental education forward, making it a valued part of foundational learning and state-wide department of education strategy.

Construct lessons and activities designed to achieve a deep understanding of environmental issues

What is a “deep understanding” of an environmental issue? A deep understanding moves beyond a student or participant communicating knowledge at the conclusion of a lesson or activity. A deep understanding includes immediate and long-term reflection on difficult concepts, with time to revisit the issue within the learner’s community or family. Activities that evoke deep understanding fall on the programmatic spectrum, meaning the learning outcomes reach beyond current curriculum or activity learning outcomes. The outcomes are, by design, long-term.

It is hard to check for lasting learning outcomes and potential behavior change after conducting a lesson in the classroom, in a one-day program, or on a field trip. Students need to process and internalize difficult concepts, especially those involving human interactions with nature, which are shaped by experiences, values, and knowledge before and after the effects of any particular environmental education lesson can be known. Simply encouraging students to discuss and debrief after a lesson on climate change, food production and consumption, forest management, or other complicated issue, may have lasting effects but those effects will not be known unless educators are able to follow up after an elapsed period of time.

Plan to follow up with learners after an elapsed period of time

In formal and non-formal learning environments it is valuable to think about the effect the lessons and activities have beyond the experience itself. Formal educators may have the advantage of having access to students for more than one year, even if they are not in their classroom. Further, some schools are designed to include thematic curricular models or systems-based thinking. Thus, conducting a brief evaluation 6 months or a year after the lesson or activity was delivered may be easy if other administrators or educators in the school are amenable to collaboration.

Non-formal educators and non-traditional educators who only see, or communicate with, their participants once could prearrange a follow up with participants, or a subset of participants. This can take the form of an email survey where people may self-select after participation, or if the group of participants are from a school or adult program, permission can be gained to ask follow-up questions after the experience.

We recognize there are many barriers within learning environments preventing future follow-up. The idea is to think beyond the immediate lesson delivery and develop methods to assess at least a few learners in the future. Not every learner need be evaluated, rather a sample of evaluative results can still be of value.

Participants will return to their community with the information about the environmental issue fresh in their minds. They may on their own ask family, coworkers, or other community members about the concepts discussed and reflect further on their own value placed on economics or their constructed environmental ethic. If they do not reflect on the experience on their own, reflective questions designed to understand and elicit an understanding of the concepts (for the learner) and viewpoints (for the educator) can be very useful to developing a stronger program or curriculum in the future. Thus, the data gathered can be formative and serve as foundational to effective activity design.

Examples of post-activity evaluation questions in formal and non-formal/non-traditional learning environments:

Environmental issue lesson and learning outcomes* Sample post-lesson evaluation Sample long-term evaluation               (6+ months- 1 year)
Formal (6-8th Grade) Forest management – Examine the balance of human-use of forest products and sound forest management. Lesson includes a trip to a local forest ecosystem. What is a healthy forest ecosystem? What forest products do we use? Can we live without them? How does the availability of the resource affect its cost? How can we use these renewable resources and not damage the ecosystem? Have you revisited a healthy forest ecosystem? What forest products have you used on a regular basis? Have you considered your use of forest products since your initial visit to the forest? If not, why not?
Non-formal/Non-traditional (Adult audience/Professional farmer or landscaper Pesticide use – Describe the ecological implications of using too many pesticides. Lesson includes a hands-on activity designed to demonstrate the broad application of pesticides. What are some strategies to mitigate short and long term problems caused by extensive use of pesticides? What changes have you made to reduce your affect on the environment from pesticide use?

Have you helped anyone to understand the potential problems?

*We will post the full 5E Lesson Plan (Forest Management by Nick Carroll) and Program Activity Agenda (Pesticide Use by Peter Warren) on the website soon.

Document the results and summarize and share findings

Creating a shared list of aligned learning outcomes, post-lesson or activity evaluation questions, and long-term evaluation methods and responses will be helpful to our environmental education community. As with any formative evaluation, having information available about what works and does not work will help us to better understand the synthesis and understanding of complicated environmental issues and lasting effects of participation. Having a list of participant learning outcome results available to you for more than one year will inform how you do the activity or lesson in the future.

Educators and learners will benefit from gaining a better understanding of longer-term learning outcomes and impacts. Knowing whether learner knowledge is retained and applied can help us to design better curriculum and programs. If we have a shared list of outcomes from teaching difficult environmental issues, together we can enhance critical thinking skills of learners of all ages.

Discussion:

What are the barriers to conducting follow up evaluations with participants in any learning environment?

In your educational setting how might you go about following up with learners after an elapsed time?

Would you be willing to share achievements from teaching controversial environmental topics from your immediate lesson or program and follow up with those participants in a few years’ time? If so, what can AAEE provide to make sharing those results easier?

Resources and Links

NAAEE’s Guidelines for Excellence Series

Cotton, Deborah R.E. (2006) Teaching controversial environmental issues: Neutrality and balance in the reality of the classroom. Educational Research. Vol. 48., (2)., 223-241. https://www.researchgate.net/publication/228785938_Teaching_controversial_environmental_issues_Neutrality_and_balance_in_the_reality_of_the_classroom

Program Development and Evaluation, University of Wisconsin-Extension

Evaluating Learning Outcomes for Citizen Science

Conservation Education Program Development, USDA

Project Learning Tree: https://www.plt.org/