2. Formative WAC Assessment using Sustainability Indicators

Definition of SIs

For the past two years, The Writing Across the Curriculum program has been working on a set of sustainability indicators (SIs) to determine the strengths and weaknesses of FAU’s WAC program. This formative assessment process was conceived in Sustainable WAC: A Whole Systems Approach to Launching and Developing Writing Across the Curriculum Programs (Cox, Galin, & Melzer, 2018) and has been refined over the past year by FAU’s WAC committee as it has refined SIs for FAU. The premise of this work is that WAC programs should take a slow, deliberate, and strategic approach to program development that includes coming to a deep understanding of campus culture and context, the use of mission and goals to guide development, the inclusion of stakeholders in determining program mission and activities, and ongoing formative assessment of program sustainability through the use of sustainability indicators.

Using SIs, program leaders can develop visual snapshots of the program at a given moment in time to identify signs of distress and success across all facets of a WAC program.

Process for WAC stakeholder group to formulate program SIs

Step 1: Brainstorming Process

1.  List all SIs that come to mind without censoring or critiquing but still focused on the goal of sustainability.
2. Qualify and narrow the list by determining if each SI is relevant, easy to understand, reliable, durable, and assessable, does not duplicate others and whether it reveal impacts as it offers historical patterns.
3. Select the 5-10 most feasible SIs by considering the resources needed to track them, relative importance, and greatest insight.
4. Unpack each SI to identify implementation procedures by determining if it can be quantified and setting the minimum and maximum thresholds (bands of equilibrium). (adapted from Bell & Morse, 2008, p. 174)

Step 2: Anthrosphere Model

We used the following figure to discuss which systems within the university to consider to develop a systematic set of SIs.

Figure 1. The WAC Anthrosphere. Reprinted from Galin, Jeffrey R. (2010), “Improving rather than proving: Self-administered sustainability mapping of WAC programs.” Council of Writing Program Administrators Conference, Minneapolis, MN.

Step 3: Mock data and figures for Writing Intensive Program

We then discussed a mock set of SI data for a writing intensive initiative that demonstrated possible SIs for such a program and the resulting radar charts that provide visual snapshots of the mock program over several years to help the WAC committee understand our ultimate goals. 

 

 Table 1 Mock WI Initiative SI Data Set

   

Mock data set for WI program for Fall 2014                       Mock data set for WI program for Fall 2016

        (Demonstrates a sustainable program)                          (Demonstrates an unsustainable program)

These radio charts represent snapshots of the mock WI program at its outset and then two years later when faculty support has dried up and there is too much demand to support sufficient sections.  The inner and outter circles in each graph represent the inner and outter bands of equalibrium, makring the boundaries of sustainability. 

Step 4. Operationalizing the SIs

The process of turning each SI into a 0-6 point scale takes some discussion with the WAC stakeholder group but can be edited and revised by a WAC leader as long as the final versions are approved by the stakeholders.  Bell and Morse (2008) describe the origin of these radar charts as “amoeba graphs” that have been traditionally used in the field of ecology to track indicator species that can signal shifts in environmental equilibrium. They suggest that these graphs can even be created by hand to obtain a preliminary sustainability snapshot. Certainly, other systems of graphing can be used, but FAU found this one especially informative because it captures all indicators in a single figure and visually represents the band of equilibrium that marks the boundaries of sustainability.

The key to translating all SIs into this scale is to start with the inner and outter bands of equilibrium represented by 1 and 5 respectively. In the sample above, 1 is represented by the minimum number of course sections needed for all desiring students to register for necessary WI courses in a semester for which there are sufficient faculty to teach them and classrooms available (200 sections). 5 represents the maximum number of sections that can be offered given the same constraints (300 sections).  Once 1 and 5 are defined, then 3 is the median of these two (250 sections). 2 and 4 get defined as the median of 1 and 3 (225) or 2 and 4 (275) respectively.  0 and 6 are defined by numbers less than or above what are sustainable boundaries (< 200 and > 300 respectively).  

SI Resources:

You may find the full set of FAU SIs with the mission and goals interesting. We will be evaluating all of them in fall 2019 and will post updates to this site as data becomes available. 

Furthermore, we are providing the above mock data as a template in Excel for any WAC program that wants to create their own radar charts by replacing our language with their own and selecting the new data range to display the new charts. Some minor resizing will likely be necessary. I recommend that each program goal (project) be represented in its own radar graph, including the program overall. 

For programs interested in using this form of formative program assessment, I recommend you read "Tracking the Sustainable Development of WAC Programs Using Sustainability Indicators: Limitations and Possibilities" (Cox and Galin, 2019), which updates significantly descriptions of this process for Sustainable WAC: A Whole Systems Approach to Launching and Developing Writing Across the Curriculum Programs  (Cox, Galin, and Melzer, 2018).