Truth, Racial Healing & Transformation Great Stories Club
Evaluation Resource Packet
This packet includes tools and resources to help you collect data for the Truth, Racial Healing & Transformation Great Stories Club (TRHT GSC) program evaluation. While only the “end of each book” surveys are required, we strongly encourage you to complete all the pre- and post-session reflections after each session. This information is useful to help you reflect throughout your project; document youth participant response; adapt your program content so that it works best for the youth; identify gaps/opportunities; and complete your final reports.
TRHT GSC Evaluation Questions
Overall, the TRHT GSC evaluation will answer the following questions:
1. What are strategies, activities, processes and content that engage youth and support their participation in all program sessions for TRHT GSC?
a. What strategies were most helpful in building meaningful relationships with youth participants?
2. For staff and youth, how effective was TRHT GSC in shifting narrative change and expanding an understanding of personal and structural racism and human equity?
3. If applicable, how were the racial healing circles implemented?
a. What was the perception of youth participants in the racial healing circles?
4. What was the structure and process of community partnerships for assisting youth in their process of racial healing?
5. To what extent has the self-perception of youth participants and library workers as agents of change shifted over the course of TRHT GSC (May 2018-April 2020)?
6. What can be improved in future TRHT GSC iterations for youth participants and library workers?
7. What is needed within the American Library Association (ALA) and beyond to support the sustainability of this work in libraries?
The data collected and submitted by you throughout the programming period will help answer some of these evaluation questions. Findings will be shared on a webinar in the summer of 2020.
PROTOCOL FOR LIBRARIES’ DATA COLLECTION FOR TRHT GSC
Please use the following evaluation protocol to guide your data collection with both the youth participants and other facilitators who are implementing the program. This packet includes:
1. (Required) TRHT GSC end of each book section survey instructions
2. (Optional) Pre- and post- individual session reflection from youth participants instructions and template
3. (Optional) Individual session reflections from youth participants instructions and template
4. (Optional, strongly recommended) Individual post-session reflection from facilitators instructions and template
5. (Optional) Overall resources from the TRHT GSC pilot for engaging youth participants
The purpose of this data collection is to:
- Explore any changes in how participants see themselves and their lives before and after the program;
- Integrate evaluative thinking to enhance each session;
- Gain insights for continuous adaptation and improvement;
- Provide you with examples for your final reports.
REQUIRED: End of Book Survey Instructions
A Qualtrics survey should be completed by facilitators at the end of each book discussion (i.e., a total of 3 surveys for three books) to determine if strategies, activities, processes and content were effective toward the goals of the program. This will also help evaluators better understand facilitator experiences, observations and lessons learned after the completion of each book. The survey can be accessed using the following link: https://become.az1.qualtrics.com/jfe/form/SV_3I8vWVSegAq1r13
OPTIONAL: Reflection Instructions
Part 1: (First and Last) Pre- and Post-Program Reflections from Youth Participants (Appendix B).
First session
Core question: Who are you today?
1) How to: To get answers on this question you can have youth participants conduct a fun activity.
2) Choice of Activities: Journaling, Drawing, Collage, Poetry, Music, Theatre, Dance, etc.
3) TAKE NOTES: Whatever activity you choose (unless it’s poetry or journaling) have youth participants write 2-3 sentences on what they chose to do and match each description to its own image. Ask: Why did you choose/create these images?
4) LABEL the work: Label the work/file “BEFORE FIRST SESSION.”
5) UPLOAD: Whether you have photos, videos or papers, you can send us the documents by uploading them into your final report. Final reports will be available starting April 1, 2019 at https://apply.ala.org/
Last session:
Core question: Who are you today?
1) Activity: Choose the same activity you did on the first day. Have youth participants revisit the activity.
2) TAKE NOTES: Whatever activity you choose (unless it’s poetry or journaling) have youth participants write 2-3 sentences on what they chose to do and match each description to its own image.
3) LABEL the work: Label the work/file “AFTER LAST SESSION.”
4) UPLOAD: Whether you have photos, videos or papers, you can send us the documents by uploading them into your final report. Final reports will be available starting April 1, 2019 at https://apply.ala.org/
5) Closing Questions: What did you like most about today’s session? What did you learn about yourself? What did you learn that you can use in your life?
6) Use the template table for the LAST session attached to take notes on the answers to these two questions (Appendix B).
Part 2: OPTIONAL: Session by Session Reflections from Youth Participants (Appendix A)
Each Session (outside of the first and last session)
At each session, youth participants should have an opportunity to tell you, and each other, what impressed or interested them and what was not effective for them. The following questions can be asked at the end of each group, except at the first and last sessions.
Closing Questions: What was your favorite part of today? What was your least favorite part of today? What did you learn that you can use in your life?
Use the template table for EACH session (Appendix A) attached to take notes on the answers to these three questions but NOT ON THE LAST SESSION. We ask that you upload these reflections to the final report shortly after each session. Final reports will be available starting April 1, 2019 at https://apply.ala.org/
Part 3: OPTIONAL (and strongly recommended): Post-Session Reflections from Facilitators
After each session, facilitators can use the Post-Session Reflection Log (Appendix C) to document what happened, what went well and what did not go well from their perspective. We ask that facilitators complete and upload their reflections to the final report shortly after each session. Final reports will be available starting April 1, 2019 at https://apply.ala.org/
Other Resources
Additional resources in this packet include findings from the pilot TRHT GSC program on effective strategies, activities, processes and content that were engaging for youth participants (Appendix D).
APPENDIX A
INDIVIDUAL SESSION REFLECTIONS FROM YOUTH PARTICIPANTS TEMPLATE
(Optional)
Complete at EACH session (EXCEPT THE LAST SESSION): Summarize with 1-2 sentences.
Session Date:
Check box if this is a session with a racial healer: □
Student Initials/Code
What was your favorite part of Great Stories Club today?
What was your least favorite part of Great Stories Club today?
What did you learn that you can use in your life?
APPENDIX B
PRE- AND POST-PROGRAM REFLECTIONS FROM YOUTH PARTICIPANTS - LAST SESSION TABLE TEMPLATE
(Optional)
Complete at LAST SESSION ONLY: Summarize with 1-2 sentences.
Student Initials/Code
What did you like most our Great Stories Club sessions?
What did you learn about yourself?
What did you learn that you can use in your life?
APPENDIX C
POST-SESSION REFLECTIONS FROM FACILITATOR TEMPLATE
(Optional)
Post-Session Facilitator Reflection Log
Please describe your activities and student engagement. Wherever you can, give examples of specific quotes youth said to illustrate your points.
Session date:
Check-in/Introduction
Describe introduction/check-in, student engagement and specific quotes that illustrate how it worked/didn’t work for youth
Activity*
Describe activity, student engagement and specific quotes that illustrate how it worked/didn’t work for youth
Check-out/Closing (if applicable)
Describe check-out/closing and specific quotes that illustrate how it worked/didn’t work for youth
Wow Moment
Things that made you go Hmmm
Oops….Better not do that again
*Note: This is any activity you chose to do with the participants, if applicable in addition to the discussion.
APPENDIX D
EFFECTIVE STRATEGIES, ACTIVITIES, PROCESSES and CONTENT FROM PILOT EVALUATION
In the pilot evaluation the following tactics helped increase youth participant engagement, continuity and passion for and contribution to TRHT GSC. Guidance was provided in terms of program structure, strategies for delivery, activities and processes. Consider using some of the content, activities, strategies and processes below to enhance your programming.
Sub Area Pilot Findings Suggestions Program Structure Design Pilot facilitators found the overall TRHT GSC design particularly useful and effective in eliciting changes in youth participants and themselves.
• Have a trained and skilled racial healing circle facilitator lead at least one session.
• Invite young people from diverse communities to meet, read and express their thoughts together.
• Support early interaction and communication between TRHT GSC facilitators and community partners. This support of strong relationship building will help to ensure a cohesive well-developed program for the youth participating. Tools Pilot facilitators noted that the resources and training were quite helpful in implementing the model and made them feel prepared. Many also developed their own tools for supporting engagement and creating a brave space.
• Provide facilitators with a diversity of resources and activities that align with the selected literature. Also make sure that adult facilitators are well-trained and supported.
• Make use of the discussion questions developed for the selected literature.
• Develop clear TRHT GSC group agreements or boundaries that every youth participant helps design and agrees to follow. Reinforce these group agreements at each session to support a safe and brave space.
Strategies for Delivery Facilitation Quality facilitation of the programs were necessary in creating the outcomes that have been achieved through the pilot. Good facilitation is characterized by inclusion, flexibility and rapport-building.
• Facilitator shares their own experiences with structural inequality.
• Come back to the text when students go off-track.
• Allow youth to contribute to the decision-making about activities to do or where the conversation goes. (e.g. “Do you want to go deeper or stay on the surface?”)
• Maintain a pattern and flow to each of the TRHT GSC sessions.
• Maintain a safe, judgment-free and brave space where conversations about difficult and complex topics can be held.
• Have some flexibility in the program sessions - so that conversation and topic discussion can flow naturally and organically. Also, if suggested program activities don't work - have the flexibility to shift to a different activity. Hosting Hosting is an art of making others feel welcomed and comfortable. It also helps to create an inclusive environment while attracting people to return.
• Show genuine concern for youth participants’ continued attendance.
• Include incentives: Have snacks every session; ask youth to choose their snacks for the next session; bring journals and pencils (at sites where no food is allowed); give out certificates at the end; bring youth “presents” like magazines and other books from home. Activities Art Art-based activities were engaging and allowed for expression through creativity.
• Ideas: Have students create a mask (great for students with writing or drawing barriers); journal, create a vision-board; make pixel art; draw; write a 6-word memoir; make videos; collage; make t-shirts; create poetry. Reflection Creating space for reflection and relaxation helped both
• Be intentional with the closing activity.
• Ideas: Have students do a privilege walk; put their thought in a Mason Jar; meditate or let their minds wander; or celebrate a friendship.
• Complete the reflection activities contained in this protocol.
participants and facilitators expand learning and help build comfort in a new space.
Historical Some pilot facilitators found that using other materials and the books was an opportunity to teach history in an engaging way.
• Ideas: Bring in additional reading—speeches, historical photos or guest speakers. Informational The use of activities also presented an opportunity to facilitate experiences around the library.
• Host a tour of the library.
Processes Discussion questions Most pilot facilitators commented on the value of the discussion questions in helping to connect the readings to the youth’s lives.
• Choose the discussion questions that you feel most connect to the events of prior weeks to help engage youth participants. Disposition Confidentiality, consistency, and playfulness is key.
• Using a funny stuffed animal or odd item as a talking piece can help keep the session playful. Relevant Questions Pilot facilitators found youth participants engaged more with the readings when they could relate it to current real-life events.
• Ask youth participants to identify ways in which real events connect to the issues in the books.