Research, Monitoring & Evaluation
What does Room to Read do to monitor and evaluate its programs?
As part of our commitment to serve children across Asia and Africa with the highest-quality programs possible, Room to Read conducts rigorous research, monitoring and evaluation to inform our program design. This includes tracking every project in an extensive Salesforce.com database, conducting independent evaluations of completed projects and performing a detailed analysis of the impact our programs have on the surrounding communities.
For each program, we have identified specific indicators to help us measure progress towards our goals. The collection and analysis of this information is managed by our Research, Monitoring and Evaluation Team, which consists of 23 full-time staff across eleven countries.
What are your future plans for RM&E?
In the coming months, we will be conducting a combination of country-specific and cross-national evaluations and research studies of our School Library, Book Publishing, School Construction and Girls’ Education programs. We will also be working to identify and refine the program indicators for the newest of our programs, Reading and Writing Instruction.
How have you used RM&E to improve your programs?
The School Libraries program in Sri Lanka provides a good example of using evaluation results to improve our programs. In 2007, the Sri Lanka team conducted a country-specific research study looking at the training provided to librarians and teachers. The study indicated that the trainings were very theoretical and needed to be more hands-on. Consequently, our team in Sri Lanka revised their training curriculum, and they have witnessed significant improvement.
Another example of programmatic adjustments inspired by our RM&E findings is that of our Girls’ Education program in Cambodia. In 2009, our team reviewed the available monitoring data and observed in increase in the drop-out rate for girls on our program as compared to previous years. The team then launched a country-specific study examining the factors contributing to drop-out. We found that many girls were leaving school to enter the workforce, and that neither their parents nor school personnel were taking an active role in encouraging them to continue their studies. As a result of these findings, our team in Cambodia began placing greater emphasis in both teacher and parent meetings on the long-term benefits of having girls’ education, in addition to stepping up mentoring for the girls through our social mobilizers (female staff members working directly with the girls).
Do you measure how students in Room to Read schools/programs do compared to other students? Do you conduct pre- and post-tests to measure program impact?
In order to compare the performance of students who benefit from our programs with that of their peers, we have built treatment-control designs into our School Library and Girls’ Education Cross-National Evaluations. These evaluations are longitudinal studies, allowing us to measure the impact of our efforts over time as well as compare them to a control group in the area.
As we develop studies to assess our programs that include skills training (such as Reading and Writing Instruction), we will seek to measure whether or not those taught skills are being acquired effectively.
How do you make sure that your schools are well-run? What happens after the funding cycle ends?
Community ownership of our projects is paramount to long-term impact, so our rigorous site selection process evaluates the likelihood that a community will sustain the facilities on its own. Then, we work to provide all the initial training and support required to help the community do just that.
In our School Libraries, for example, we provide three years of ongoing training to school officials, librarians and teachers on the effective use and management of a library. In 2011, we began a post-completion study, designed to assess how the libraries that have “graduated” from our support period are functioning on their own. View an infographic containing Phase I results from the School Libraries Post-Completion Study.
How do you know if children are reading the books in the School Libraries?
We conduct regular evaluations of our School Libraries to monitor their use and look for areas of potential programmatic improvement. In these evaluations, both teachers and students have consistently reported that students’ skill level and frequency of reading had improved as a result of their new library.
How does Room to Read ensure that local language books are child-friendly and age appropriate?
Our staff collaborates with local authors, illustrators, writing organizations, and printers to publish new children's books each year. After we source and develop the content, our Book Development Committees, made up of individuals with publishing expertise, assist with editing and refinement of the manuscripts.
Although the attrition rate in your Girls' Education program is impressively low, some girls drop out. Why?
Our attrition rate—5 percent in 2011—is much lower than the average for girls in the countries where we work. That said, we do occasionally see some dropout for a variety of reasons.
In many developing countries, families depend on girls for labor. Although social attitudes are changing over time, many parents are not aware of the benefits of educating girls, so helping their daughters stay in school is not a priority. Our program includes parental engagement activities, which help communicate the importance of educating all children, but it does not provide a 100 percent guarantee that a girl’s family will support her education.
How do your program goals fit into the Millennium Development Goals?
Our goals and objectives are most closely aligned with two of the United Nations Millennium Development Goals: achieving universal primary education; and promoting gender equality and empowering women. Beyond that, we know that improving access to education also indirectly promotes all of the other Millennium Development Goals.
How does Room to Read estimate the total number of children benefitted by its programs?
As Room to Read has increased its global footprint, we are often asked how we calculate the number of children our programs have benefitted? Here's how we do it:
To estimate the number of children for whom we provide access to educational resources, we take into consideration several factors. We count the total number of children provided access to our School Libraries over an eight year period; our School Construction projects over ten years; and all participants in our Girls’ Education program. These figures are based on school enrollment during the first year of the program and the estimated enrollment of new classes thereafter. All enrollment figures are collected by our in-country staff through surveying and monitoring visits. Because we often offer multiple programs at the same project site, we also account for program layering to avoid double-counting.