Youth in Evaluation week: Engaging young people in a Formative Evaluation of UNICEF’s Gender Transformative Programming in Eastern and Southern Africa

This week is Youth in Evaluation week. Convened by the Eval4Action advocacy campaign, its regional partners and the EvalYouth Global Network, Youth in Evaluation week aims to focus global attention on advancing the meaningful engagement of youth in evaluation. This blog shares SDDirect’s reflections on engaging young people in one of our current evaluations.

SDDirect has been contracted by UNICEF’s Eastern and Southern Africa Office to conduct a Formative Evaluation of Gender Transformative Programming through Investment in Adolescent Girls’ Leadership in Eastern and Southern Africa. The purpose of the evaluation is to understand how current programming, investments and practices to invest in adolescent girls’ leadership, voice and agency are shaping UNICEF’s gender transformative programming approach in the region over the coming years. The methodology includes in-country and desk-based case studies, a regional staff survey, and the use of our Gender Equality and Social Inclusion (GESI) continuum. We have now finished the inception phase and are starting data collection.

We are engaging young people in the evaluation to ensure their voices, views and opinions are embodied in our methodology and approach, as well as the findings, conclusions, and recommendations. Young people hold three main roles:

1. Regional Adolescent Advisory Group (RAAG): with UNICEF’s assistance, we set up a group of 18 young people from 10 countries in the region (aged 15–19) to ensure that we are grounding the evaluation methodology and findings in the lived experiences of young people. Their role is to:

   • Share their thoughts on the evaluation questions to ensure they are relevant to young people and reflect young peoples’ perspectives.
   • Share ideas on the best way to engage young people as key informants throughout the evaluation.
   • Provide insights and validate our findings and recommendations to ensure relevance to different contexts and increase credibility.
   • Enhance the evaluation’s accountability to young people themselves.

    

2. Adolescent Panels: In each of the three countries where we are conducting in-country case studies (Mozambique, Namibia and Uganda), we are establishing an Adolescent Panel of four young people. Adolescent Panellists will be trained by and work closely with our in-country experts to collect views and insights from their peers and will participate in sensemaking workshops to share the findings from their fieldwork.

3. Key informants: In Mozambique, Namibia and Uganda, the in-country experts and Adolescent Panellists will conduct primary data collection with a small number of young people involved in UNICEF programmes, mostly as part of group discussions.

What we have learnt so far

We held our first meeting with the RAAG during the evaluation’s inception phase and since then have also been engaged ad-hoc with the young people involved via a WhatsApp group and email. Here are some of the things we have learned so far:

1. Importance of diversity within the RAAG: Having an intentional focus on diversity within the RAAG was critical, and our interviews with UNICEF staff during the inception phase further reinforced the importance of this resolve. They noted that often there are missed opportunities with ‘harder to reach’ groups (e.g., adolescents with disabilities, out of education, living in rural areas, pregnant girls and young mothers, girls living with HIV), who are less likely to participate in adolescent groups and programming, without ensuring an intentional focus on their inclusion. This can lead to the unintentional platforming of the same types of ‘youth activists’, missing out on key voices and only nurturing an ‘elite’ group of young people.

   How did this learning inform our evaluation? We asked UNICEF Country Offices to nominate young people for the RAAG, encouraging diversity in terms of gender, geographical location socio-economic background, religion, education, abilities, children’s interests, and country context. 18 adolescents were nominated, all between 15 and 19 years-old and including several from ‘harder-to-reach’ groups¹. We hope this will ensure that the evaluation is grounded in diverse adolescent experiences and voices.

2. Meaningful engagement and participation of young people: During inception we considered how we would ensure meaningful engagement and participation of adolescents, avoiding tokenism, and asked the RAAG for their thoughts.

   How did this learning inform our evaluation? We understood that ‘meaningful participation’ means involving young people in decision making and ensuring they get something back from their participation. We are committed to make efforts to ensure they see the value of their involvement, and we remain accountable to them. We established the RAAG during the inception phase to include young people from the offset, we are setting up meeting times outside of school hours, using ‘adolescent friendly’ terminology when engaging with them and setting up a WhatsApp group for easy exchange on a platform they are already familiar with.

3. Communicating effectively with young people. During the first meeting of the RAAG, we sought their advice about how best to obtain the views and opinions from other young people about their experiences of engaging with UNICEF. Some of their advice to us included:

   • Building rapport with young people before asking them questions to inform the evaluation, through questions about their daily lives and likes/dislikes.
   • Asking questions in a group setting rather than individually, as young people would feel more comfortable. However, they warned we should be mindful of ‘group think’, and peer pressure, as some young people may not want to challenge the opinions of their peers.
   • Engaging young people creatively – some might like to write poems/do role plays/draw pictures rather than just talk and answer questions.
   • Zoom is not always accessible by young people, so we should offer a variety of platforms.
   • When communicating back the findings to young people, it is important to ensure and think about what young people would feel captivating. Videos were a popular choice, as were posters, social media, and using other types of community group feedback platforms.

    

   How did this learning inform our evaluation? The RAAG’s feedback is informing our data collection tools in terms of the questions we ask and the format. We are adapting our ways of communication to ensure accessibility. For example, one member of the group is visually impaired so we will be sharing documents and content through videos with voice over and voice notes as well as in written form. After our initial meeting with the RAAG, we created a video for the group, summarising what we had discussed and how their feedback informed the evaluation. Please watch it here:

We also shared a summary document and updated the group via WhatsApp, using multiple communication styles to increase inclusivity.

We will meet with the RAAG twice more at key stages during the evaluation, to enrich our findings and ensure our recommendations are grounded in their reality and context. We will continue to share with them how their inputs are shaping the evaluation, through periodic messages and updates on the WhatsApp group and via email.  The final evaluation report will include an ’adolescent friendly’ version, to ensure that adolescents can engage with it.

¹ The group is made up of 14 adolescent girls and 4 adolescent boys, with an even spread across age groups. One RAAG member is a teenage mother, two are living with a disability, and several are from rural locations.