Great Student Stories: Part 1
Stories open up our world—they make us laugh, they make us cry, and sometimes they even give us new insights about others and ourselves. Stories are part of every culture and help us make sense of the world.
As I wrote in a previous post, stories can also help programs and organizations make sense of their worlds. When collected with rigor, information gathered through stories can be used to influence policies, develop programs, and measure effectiveness. Sometimes stories are the best way to gather this information.
I recently turned theory into practice with a method I developed for collecting evaluation stories from elementary and middle school students. My goal was to help the AmeriCorps Partnership for Student Achievement program in the Forest Grove school district find a practical evaluation method for measuring student outcomes.
Best Outcomes Hardest to Capture
For the past several years, Forest Grove has received funding to place 20 AmeriCorps members each year in district schools as tutors and mentors in the Partnerships for Student Achievement (PSA) program. Each AmeriCorps member works one-on-one with several students throughout the school year in an effort to bring them up to grade level requirements in reading, math and English.
As the evaluator hired to help Forest Grove structure a meaningful PSA evaluation for the 2011-12 school year, I started by reviewing the existing evaluation plan. I found components for tracking numbers and types of students, for monitoring and assessing AmeriCorps member activity and for collecting pre/post assessment data on student achievement in math, reading and English from teachers. I was impressed with the comprehensiveness of the PSA evaluation plan. But I also realized that a process for eliciting direct feedback from program recipients–the students–was missing.
It’s not that Forest Grove officials didn’t think it was important to include student feedback in the evaluation. To the contrary: active student reflection with teachers and members was a stated program goal and genuinely encouraged. But the people at Forest Grove found themselves in the same dilemma I often witness among nonprofits: their best program outcomes were the hardest to measure.
A Qualitative Approach
After listening to what Forest Grove officials had to say about program goals and the realities of implementing a student data collection effort on the ground, it was clear that the situation called for a qualitative approach. But it needed to be a practical approach that could be implemented without additional funding. It also needed to be one that would harvest rich student outcome data in an objective and reliable manner without intrusion or additional requirements on teacher time.
Knowing these realities (and the dedication of PSA staff and their funders to a quality evaluation), I suggested collecting stories from students through semi-structured interviews conducted by AmeriCorps members. Not only would the stories engage students in reflecting on their experience, they would enlist AmeriCorps members in learning about evaluation (a skill they are encouraged to build). The stories would also complement the numeric data collected with quantitative strategies and enhance the meaning of that data. The program director loved the idea!
Beyond the evaluation benefits, the director envisioned the stories building student self esteem in the telling of them and pride in academic and personal achievements. She also felt the stories would be useful for program promotion and attracting additional funding. It prompted her to start calling them “Great Student Stories.”
Story Making
To help AmeriCorps members compose a story that would address key evaluation questions, I designed an interview guide (Great Student Stories) to structure the dialogue with the student. Throughout the upcoming school year, AmeriCorps members will be collecting stories from three of the many students they mentor. The students they select must be those with whom they: (1) have worked extensively throughout the year, (2) have seen considerable progress, and (3) feel have a reasonable ability to articulate their experience.
When writing the story up, the AmeriCorps member will weave in relevant information from direct experiences s/he has had with the student as well as pertinent information garnered informally from conversations with teachers and parents throughout the year. This creates a broader context in which to situate the student story. Ultimately, the member crafts a one-page story containing background information, the student’s story, and their own reflections on the student’s process.
At the end of the year, the PSA advisory committee selects the most compelling stories to incorporate in the evaluation report. When funders read the report they will learn not only how many students were served and by what types of services, they will get a front row seat to the lived experiences of those students. Some of the stories may make them laugh, some may even make them cry. What’s certain though is that every story will put a human face on the program they’re funding.
Carlos’ Story
Stay tuned for my next blog where you’ll meet Carlos, a 4th grade Forest Grove student who struggles with math and sitting still. I wrote his story (based on true events) as an example of the evaluation stories that AmeriCorps members in Forest Grove will be composing this upcoming school year.
Thank you for reading my blog. Just enter your email address below to become a subscriber. It’s free and you’ll receive a notice every time I post.
