Indonesia’s PNMP Generasi Program: Final Impact Evaluation Report

Asia

Indonesia

Social Protection

Conditional Cash Transfers (CCT)

Olken, B. A., J. Onishi, and S. Wong

2011

Olken, B. A., J. Onishi, and S. Wong. 2011. Indonesia’s PNMP Generasi Program: Final Impact Evaluation Report. The World Bank, Jakarta, Indonesia.
View publication

-

2011

PNMP Generasi Program

2007-2010

Improving access to basic quality health and education services

See section below on sample selection

Each year, villages receive a block grant. With the assistance of trained program facilitators and service delivery workers, villagers undertake a social mapping and participatory planning exercise to decide how best to use the block grant funds to reach 12 education and health targets related to maternal and child health behavior and education behavior (such as prenatal and postnatal care, child immunizations, and primary and junior secondary school enrollment and attendance). To give communities incentives to focus on the most effective policies, the government bases the size of the village’s Generasi block grant for the subsequent year partly on the village’s performance on each of the targeted indicators.

• Children’s work: Wage work and household chores (extensive margin and hours worked) for 5-17 year old children
• Education 1: Primary school enrollment and attendance (for 7-12 year olds) and junior secondary school enrollment and attendance (for 13-15 year olds)
• Education 2: Performance on mathematics and language tests (7-12 and 13-15 year olds)

Cluster randomized control trial

Sample: Generasi locations were selected in 4 steps:

• Step 1: Five target provinces were identified by the government.


• Step 2: Within these provinces the wealthiest 20% of districts (in terms of the poverty rate, malnutrition rate, and junior secondary school transition rate) were discarded.


• Step 3: Twenty districts were randomly selected from the remaining districts, stratified by island group.


• Step 4: Within these districts 300 target subdistricts were identified, targeting poor, rural areas that had an existing community-driven development infrastructure.

Identification strategy: The 300 subdistricts were randomly assigned by computer into one of three equal-sized groups: “incentivized” treatment with the pay-for-performance component described above (100 subdistricts), “non-incentivized” treatment without the pay-for-performance incentives (100 subdistricts), or control (100 subdistricts). Within subdistricts, all villages received the same treatment.
Survey instrument: A three-wave household survey was implemented in 8 randomly selected villages in each of the subdistricts. Wave I, the baseline round, was conducted from June to August 2007 prior to Generasi implementation. Wave II, the first follow-up survey round, was conducted from October to December 2008. Wave III, a longer term follow-up survey round, was conducted from October 2009 to January 2010.
The sampling design for the household component of the Generasi surveys was chosen to ensure adequate coverage in three key demographic groups: mothers who recently were pregnant or gave birth, children under age 3, and children of school age. Within each village, one hamlet was randomly selected, and a list of all households was obtained from the head of the hamlet. Five households were randomly sampled from that list to be interviewed. These households were stratified so that two selected households had at least one child under age 2, two selected households had a child under age 15 but no children under age 2, and one household had no children under age 15.
In the follow-up surveys, in half of the randomly selected villages (four villages out of the eight villages sampled in every subdistrict), the same households sampled in Wave I were contacted again in subsequent waves to form an individual level panel. Teams tracked and re-interviewed migrated or split households who provided information for any of the married women or children modules, as long as they were within the same subdistrict. In panel areas, 95 percent of target households were able to be re-interviewed in Wave 2 and 98 percent of target households were able to be re-interviewed in Wave 3. In the other half of villages, a new cross-section of households was drawn in each survey wave.

When the authors compare children in intervention and control villages (without distinguishing between “incentivized” and “non-incentivized” treatment) they find a significant decrease in the enrollment and attendance of 13-15 year olds and no significant impact for 7-12 year olds in wave II. In wave III this negative effect is no longer present and the authors find a significant increase of 8 percentage points in the school enrollment of 7-12 year olds, while the attendance of 7-12 year olds and the enrollment and attendance of 13-15 year olds have not changed significantly. Consistent with the decrease in enrollment and attendance, in wave II there is evidence that the program significantly increased the hours 7-15 year old children spend in wage work (0.18 hours) and in household chores (0.67 hours). Unfortunately, the text provides no reference for the reference time period, but we assume these are either reductions per day or per week. There is no evidence that the program affected the extensive margins of wage work in wave II. Nor is there evidence of a significant impact on the extensive and intensive margins of wage work and household chores in wave III. When the authors compare the impact of “incentivized” and “non-incentivized” treatment they find that the significant decrease in the wave II enrollment and attendance of 13-15 year olds is equally strong in both treatment arms. The increase in the school enrollment of 7-12 year olds in Wave III comes from the “non-incentivized” treatment arm. This comparison is not carried out for wage work and household chores. The project does not appear to have had an impact on learning of school-aged children, measured by math or language tests administered separately for children in the 7–12 and 13–15 age groups.

Child Labour

Households

National

There is some evidence that the program increased the hours 7-15 year old children spend in wage work (0.18 hours) and in household chores (0.67 hours).

There has been a decrease in the enrollment and attendance of 13-15 year olds and no significant impact for 7-12 year olds in wave II. In wave III this negative effect is no longer present and the authors find a significant increase of 8 percentage points in the school enrollment of 7-12 year olds, while the attendance of 7-12 year olds and the enrollment and attendance of 13-15 year olds have not changed significantly. When the authors compare the impact of “incentivized” and “non-incentivized” treatment they find that the significant decrease in the wave II enrollment and attendance of 13-15 year olds is equally strong in both treatment arms. The increase in the school enrollment of 7-12 year olds in Wave III comes from the “non-incentivized” treatment arm. This comparison is not carried out for wage work and household chores. The project does not appear to have had an impact on learning of school-aged children, measured by math or language tests administered separately for children in the 7–12 and 13–15 age groups.

A complication in the evaluation design is that the government, for budgetary reasons, prioritized locations that had previously participated in the PNPM rural infrastructure program. This issue appears to have affected the randomization procedure. For more information, we refer the reader to the report.