The Latest

Measuring the Effectiveness of a Life Skills Education

November 03, 2016

In 2015, Room to Read launched its Girls’ Education Program evaluation in India, funded in part by the U.S. Department of Labor, Echidna Giving and the Bill & Melinda Gates Foundation.

Besides aiming to measure the effectiveness of the Girls’ Education Program in improving a range of outcomes, including academic performance and the time participants invest in education and work, the evaluation seeks to fill a critical gap in our knowledge by rigorously assessing girls’ development of life skills, which include confidence and self-esteem as well as such competencies as decision-making and perseverance.

“A focus on non-cognitive or life skills is an area that has received a lot of attention in the U.S. because we’re realizing these skills can have a significant impact on a student’s welfare, but this is a relatively new concept in the developing world.” Jessica Leight Assistant Professor of Economics, Williams College

“A focus on non-cognitive or life skills is an area that has received a lot of attention in the U.S. because we’re realizing these skills can have a significant impact on a student’s welfare,” noted one of the evaluation’s principal investigators, Williams College assistant professor of economics Jessica Leight. “But this is a relatively new concept in the developing world. There isn’t a lot of evidence beyond the anecdotal, so our aim is to contribute solid research to it.”

Using a randomized controlled trial (RCT) design, considered the gold standard in scientific research, the evaluation compares girls from two different groups of schools in India. Schools were randomly assigned to either the program or control group in order to allow for unbiased comparisons. The program group comprises 60 schools whose female students are enrolled in Room to Read’s Girls’ Education Program. The control group is a different set of 59 schools nearby whose female students are not in the program and are not receiving any kind of life skills training.

But how exactly does a scientist measure life skills? Leight’s team faced a number of challenges. “Comprehension is a problem. Different girls will perceive a particular question in different ways,” she noted. “Another challenge was to develop concrete questions that the girls can relate to their daily experiences. It’s not easy to probe their aspirations around careers, for example, when they have typically only been exposed to a small number of women working.”

The team also needed to ensure that the questions asked by the study generated a range of responses. “If we simply ask how the subject feels and everyone answers positively, then we haven’t learned much from that question,” Leight explained. “We want to structure questions in such a way that the girls feel comfortable selecting any one of a number of options to reflect their own personal experiences.” 

An additional challenge was developmental. In India and most other Room to Read countries, lower secondary school (and our program) starts in the sixth grade, and abstract concepts like self-esteem and perseverance can be hard for 11- and 12-year-olds to grasp. “When you ask a girl, ‘How do you react when you encounter an obstacle?’ it may be hard for her to understand the question,” said Leight. “Questions needed to be made as concrete as possible, so that girls can relate them to specific circumstances or behaviors in their own lives.” 

Finally, Leight’s team needed to ensure that they are measuring what they intended to measure. “In answering the questions, the girls may just repeat what they learned in the life skills lessons,” she explained, “while the girls in the control group didn’t hear the lessons so they would not be able to repeat them. That means we may just be picking up on exposure to life skills education.” The team’s solution was to supplement the questionnaire with more objective measures, which is how they came up with the mirror task and the scavenger hunt.

In the mirror task the girls are asked to draw a series of shapes by following their reflection in a mirror. The shapes become progressively harder to draw with the last shape being nearly impossible to reproduce accurately. “We see how long she is willing to do this, testing her perseverance,” said Leight. In the scavenger hunt the girls receive a list of things to obtain, such as an item of clothing or food. “She’s encouraged to borrow them or find them at home,” Leight explained. “The girls might ask family, friends, or others they know. It shows how well girls can advocate on their own behalf.”

Global Monitoring Report, Girls' Education, India

A girl performing the mirror challenge in India.

These two tasks may seem like pure fun and games—and from what we’ve seen so far, girls have a good time completing them! But there is more here than may initially meet the eye. Because these measures ask girls to put key skills like perseverance and negotiation into practice, they are objective in a way that simple self-report questions can’t be. There is no way for a girl to make it up, or to simply tell her interviewers what she thinks they want to hear. And thanks to our rigorous RCT-based design, we can make a compelling case that any systematic difference we find between program participants and non-participants in the performance of these tasks is the result of our program. 

In 2015 the team piloted the evaluation and selected the schools for the sample. Interviews with all 2,500 girls participating in the study began in early 2016. The evaluation will be ongoing through 2018.

Room to Read has already begun adapting the measures that Leight and her team developed in order to create a life skills assessment tool for our own ongoing use outside the evaluation and after it ends. Developing this internal tool presents additional challenges: it will need to be adjusted to meet the needs of a wider range of ages, and across a wider variety of cultural contexts. Certain particularly time-consuming or costly elements that work well in a large-scale evaluation will also need to be reduced or eliminated for use in ongoing monitoring, but this must be carefully balanced against the need to retain the reliability and objectivity of the evaluation tool. We have initiated our first phases of pilot testing for this work, and will report on our progress in next year’s Global Monitoring Report.

***

This story is part of our 2015 Global Monitoring Report. Check out  the full online report here.

Download the full report (PDF, 11 MB)

 

Sushila's candid thoughts on her life skills evaluation

Related Articles

How Roshanara Changed Her Story


December 12, 2016

Girls' Education, India

Read More >

Proud Father Invests in his Daughter’s Dreams


December 05, 2016

Girls' Education, India

Read More >

The Nonprofit Using Scavenger Hunts to Research Girls' Education


November 21, 2016

Girls' Education, India

Read More >

Vietnam’s Girls’ Education Class of 2014: Where Are They Now?


November 03, 2016

Girls' Education, Asia

Read More >

A Veteran Teacher Perfects Her Style


November 03, 2016

Literacy, Asia

Read More >

Multiyear Strategizing at This Year’s Country Management Conference


October 18, 2016

Literacy, India

Read More >

How Her Mother Inspired Nirupa to Beat the Odds


May 08, 2016

Girls' Education, Asia

Read More >

A Year After the Nepal Earthquake, a Survivor Finds Strength at School


April 22, 2016

Girls' Education, Asia

Read More >

For India’s School Girls, How One Father’s Support Gets Multiplied


March 14, 2016

Girls' Education, India

Read More >

What Happens When You Take Girls’ Education in Laos Beyond the Classroom?


January 27, 2016

Girls' Education, Asia

Read More >

John Wood: My Three-Phrase Philosophy for Changing the World Through Education


John Wood

| November 13, 2015

Literacy, Asia

Read More >

#ThankstoEducation I Won’t Take ‘No’ for an Answer


November 11, 2015

Literacy, Asia

Read More >

#ThankstoEducation We Can Help Our Children Reach Their Dreams


November 09, 2015

Girls' Education, Asia

Read More >

#ThankstoEducation I Can Teach Girls the Skills They Need to Make Key Life Decis...


November 06, 2015

Girls' Education, India

Read More >

#ThankstoEducation I Can Change My Family’s Destiny for the Better


November 04, 2015

Girls' Education, Asia

Read More >

International Day of the Girl: Meet Shabnam Whose Courage Has Inspired Hundreds ...


October 09, 2015

Girls' Education, India

Read More >

She Didn’t Know It Was Her Wedding Day


May 07, 2015

Girls' Education, Asia

Read More >

The Results Are In: Libraries Matter.


March 14, 2015

Literacy, India

Read More >

Now This Girl Knows How to Handle a Street-Harasser


February 24, 2015

Girls' Education, India

Read More >

After Devastating Flood in Nepal, Room to Read Team Bands Together to Bring Reli...


January 28, 2015

Girls' Education, Asia

Read More >

DO NOT READ THIS.


January 25, 2015

Publications & Reports, Asia

Read More >

The Right to Choose By Erin Ganju, co-founder and CEO Room to Read


Erin Ganju

| January 21, 2015

Publications & Reports, Asia

Read More >

VIDEO: The story of a girl worth celebrating on this International Day of the Gi...


October 10, 2014

Girls' Education, Asia

Read More >

Erin Ganju: Flaunt Your Flaws: Why NGOs Need Transparency to Thrive


April 04, 2014

In the News, Asia

Read More >

Show and Tell: Boys and Girls Celebrate Women in Laos


March 20, 2014

Girls' Education, Asia

Read More >

International Women’s Day: Dreaming of Olympic Gold in Nepal


March 04, 2014

Girls' Education, Asia

Read More >

Reconnecting with "Nurse Prem" in Nepal


December 16, 2013

Girls' Education, Asia

Read More >

Guest Blog: Atlassian Foundation Sees Their Investments at Work in Cambodia


December 11, 2013

Girls' Education, Asia

Read More >

A "Shining Moonlight" For Girls Everywhere


November 25, 2013

Girls' Education, Asia

Read More >

Show and Tell: Innovating for Girls’ Education


October 14, 2013

Girls' Education, Asia

Read More >

In Honor of International Day of the Girl Child: From Nepal to New Jersey


October 07, 2013

Girls' Education, Asia

Read More >

In Laos, Life Skills Training and Parental Support Help Sisters find Strength


August 12, 2013

Girls' Education, Asia

Read More >

In Honor of Malala Day: The Power of Perseverance


July 12, 2013

Girls' Education, Asia

Read More >

Guest Blog by Author Manjushree Thapa: Meeting Some of Nepal's Extraordinary Gir...


June 21, 2013

Girls' Education, Asia

Read More >

Tune in: Suma’s Story Comes to CNN this Sunday!


June 13, 2013

Girls' Education, Asia

Read More >

Social Mobilizers in India and Vietnam Get a Boost from Technology Pilot


May 20, 2013

Girls' Education, India

Read More >

"A Mother's Day Tribute: The Role of Women and Parenting in Girls' Education Aro...


Erin Ganju

| May 09, 2012

Girls' Education, India

Read More >