Term of Reference
Final Evaluation of the USAID-funded Cross Sectoral Strategies for Climate Change and Disaster Risk Reduction (CADRE) Program
I. Background
Funded by the United States Agency for International Development (USAID), the Cross Sectoral Strategies for Climate Change and Disaster Risk Reduction (CADRE) in Indonesia presents a strategic collaboration between Project Concern International (PCI), KabaHill Centre, universities, consultants, local communities and government to address the needs of 30 communities in three districts – Bengkulu City, North Bengkulu and Kaur – corresponding to an estimated 35,000 people in Bengkulu Province, one of the most disaster-prone provinces in the country. Specifically, the project seeks to achieve strengthened resilience of vulnerable populations in Bengkulu Province to disaster and climate change. This will be achieved through the following three intermediate results:
IR 1. Strengthened institutional capacity for disaster and climate
change management. PCI will build the capacity of three district
disaster management bodies (BPBDs) and facilitate the formation of 30
Community Disaster Risk Reduction Committees (C/DRRCs) to assess and map
risks, develop and initiate Disaster Risk Reduction action plans. By
the end of the project, these committees will have been formed, risks
mapped and quantifiable achievements made against their action plans.
IR 2. Reduced exposure and vulnerability through adaptive livelihoods. The project will directly support rural households affected by disaster and climate change to proactively begin to adapt livelihoods (i.e. incorporation of crops and methods that build resiliency to disaster and climate change). This will be achieved through the formation of 30 adaptive livelihood groups of 300-450 farmers who will be trained in the principles of permaculture and provided with technical assistance to access capital. By the end of the project it is expected that at least 75% percent of participants will be incorporating adaptive methods into their livelihoods with more than $90,000 leveraged in capital to support these activities.
IR 3. Improved disaster preparedness practices of communities through knowledge and education. The project will improve knowledge and education among targeted communities in several disaster preparedness competencies, including early warning readiness and disaster preparation and response. PCI utilizes a variety of education and knowledge building activities including community radio, home visits, traditional theatre, art and other forms of community education. By the end of the project, all communities will have developed community-based early warning systems, conducted drills and simulations, and at least 50% of households will have a planned response in case of a disaster.
In summary, key intervention areas are the following:
II. Objective of Evaluation
This evaluation shall address the achievement of the program’s goal to strengthen resilience of rural communities to natural disasters in three districts (Bengkulu City, North Bengkulu and Kaur) in Bengkulu province, and the effectiveness of the program design in achieving the program goal. The detailed project monitoring plan (PMP) with performance indicators is attached.
The evaluation should address the following questions. These should serve as guidelines. The external consultant is expected to refine and adapt the final research questions based on consultations with the CADRE team.
2.1. Overall Program Outcomes and Impact
2.1.1. Did the program meet targets and goals related to the performance indicators defined in the Performance Monitoring and Evaluation Plan? What were the reasons that key performance targets were or were not met?
2.1.2. To what extent did the program strengthen institutional capacity for disaster and climate change management among BPBDs and C/DRRCs? What have BPBDs and C/DRRCs achieved as a result of increased capacity?
2.1.3. To what extent was the program able to reduce exposure and vulnerability to disaster by increasing technical knowledge of, adoption, and income from adaptive livelihoods?
2.1.4. To what extent did the program improve disaster preparedness practices among targeted communities through knowledge and education? What were the major factors influencing the adoption of improved disaster preparedness practices?
2.1.5. To what extent did program interventions impact women beneficiaries in addressing gender-specific needs to climate change adaptation (CCA)? How did these interventions help women respond to shocks and stresses to mitigate their risks?
2.3. Relevance of the Design and Program Coverage
2.3.1. Was the program design relevant to the targeted beneficiaries? What are the key determining factors that make this program design relevant?
2.3.2. What components of the project design had a positive impact on DRR and CCA, and what components had little/no impact or could have been improved? Components include: community mobilization, constitution of DRM committees, approach to adaptive livelihoods, linking groups with other stakeholders, partner performance, and technical service provision to groups
2.3.3. What is the current program coverage for improving CCA and DRR knowledge and practices? Are there any significant differences across geographic areas?
2.3.4 . Is the program design replicable and scalable? If so, what factors will determine where this program can or cannot be replicated and scaled?
2.4. Sustainability
2.4.1. Are the groups formed during the program (DRR groups, Livelihood groups, and organic farming groups) functioning independently?
2.4.2. Have the above mentioned groups formed linkages to the available institutional and financial resources? If linkages have been formed with available resources, what outcomes have been generated, and are the linkages sustainable?
2.4.3. Have beneficiaries created any innovations based on the adaptive livelihood trainings that could make them more sustainable?
2.4.4. Did the adoption of adaptive livelihoods have any negative impacts that would affect their sustainability?
2.4.5. What is the current policy environment influencing the likelihood of sustainability for these groups?
2.4.6. Has the program created any “champions” who can continue to lead the process of developing resilient communities?
Through the process of responding to the research questions, the external evaluation consultant is expected to identify lessons learned, challenges and unexpected results to provide concrete recommendations for improved future program design and implementation. The consultant will also document best practices to provide recommendations for organizational learning.
III. Evaluation Design: Sample, Data Sources and Research Methods
Sample Population
he evaluation consultant is expected to propose an adequate sample that will respond to the proposed research questions. The sample should take into consideration all of the following stakeholders:
Both primary and secondary data sources should be considered in the development of the program evaluation design. Primary data sources should consider the entire CADRE population and appropriate respondents/key informants. Secondary data sources include program source documents, e.g. CADRE contract agreement, work plans, reports (baseline, mid-term, annual, etc.), policy documents, etc.
Research Methods
A mixed-methods approach for collecting both quantitative and qualitative data is preferred. Data should be representative of the program intervention and can include document reviews and synthesis, focus groups, key informant interviews, observations, surveys, among other kinds of research methods.
Quantitative data should be managed using appropriate data management technology and analyzed using statistical software (e.g. R, SPSS, STATA, Epi-Info, etc.). Quantitative and qualitative data should be presented using the appropriate data visualizations to effectively represent key results.
IV. Application Instructions
Applicants must submit a cover letter, CV, 3 professional references, and a USAID Biodata 1420 form to be considered for the consultancy. Submit your application to Arief Firdaus, mafirdaus@pci.or.id, with a copy to hmaad@pci.or.id with “Application-CADRE Final Consultancy” in the subject line by 21 Nov 2014.
V. Requirements for the External Evaluation Consultant(s):
The person or firm conducting the evaluation should meet the following criteria:
For those who are qualified and interested in this consultancy work, please submit your technical proposal along with budget and CV/resume to: Arief Firdaus, mafirdaus@pci.or.id with a copy to hmaad@pci.or.id and Indonesia CADRE Final Evaluation as the subject of your email.
Latest date for proposals will be on 28 Nov 2014. Only shortlisted consultant will be contacted for recruitment
Final Evaluation of the USAID-funded Cross Sectoral Strategies for Climate Change and Disaster Risk Reduction (CADRE) Program
I. Background
Funded by the United States Agency for International Development (USAID), the Cross Sectoral Strategies for Climate Change and Disaster Risk Reduction (CADRE) in Indonesia presents a strategic collaboration between Project Concern International (PCI), KabaHill Centre, universities, consultants, local communities and government to address the needs of 30 communities in three districts – Bengkulu City, North Bengkulu and Kaur – corresponding to an estimated 35,000 people in Bengkulu Province, one of the most disaster-prone provinces in the country. Specifically, the project seeks to achieve strengthened resilience of vulnerable populations in Bengkulu Province to disaster and climate change. This will be achieved through the following three intermediate results:
IR 2. Reduced exposure and vulnerability through adaptive livelihoods. The project will directly support rural households affected by disaster and climate change to proactively begin to adapt livelihoods (i.e. incorporation of crops and methods that build resiliency to disaster and climate change). This will be achieved through the formation of 30 adaptive livelihood groups of 300-450 farmers who will be trained in the principles of permaculture and provided with technical assistance to access capital. By the end of the project it is expected that at least 75% percent of participants will be incorporating adaptive methods into their livelihoods with more than $90,000 leveraged in capital to support these activities.
IR 3. Improved disaster preparedness practices of communities through knowledge and education. The project will improve knowledge and education among targeted communities in several disaster preparedness competencies, including early warning readiness and disaster preparation and response. PCI utilizes a variety of education and knowledge building activities including community radio, home visits, traditional theatre, art and other forms of community education. By the end of the project, all communities will have developed community-based early warning systems, conducted drills and simulations, and at least 50% of households will have a planned response in case of a disaster.
In summary, key intervention areas are the following:
- DRR capacity building at the community, government and school level, including the formation of C/DRRC at the village level
- Information dissemination through various channels, including social group activities, religious leaders, school-based activities, and radio
- DRR practices through the development of early warning systems and disaster simulations
- Permaculture training and Farmer Field Schools
- Small economic development activities for alternative livelihoods such as food processing, livestock, and catfish breeding
- Advocacy and networking with stakeholders through coordination meetings, seminars and workshops with government agencies, universities, and local entrepreneurs
II. Objective of Evaluation
This evaluation shall address the achievement of the program’s goal to strengthen resilience of rural communities to natural disasters in three districts (Bengkulu City, North Bengkulu and Kaur) in Bengkulu province, and the effectiveness of the program design in achieving the program goal. The detailed project monitoring plan (PMP) with performance indicators is attached.
The evaluation should address the following questions. These should serve as guidelines. The external consultant is expected to refine and adapt the final research questions based on consultations with the CADRE team.
2.1. Overall Program Outcomes and Impact
2.1.1. Did the program meet targets and goals related to the performance indicators defined in the Performance Monitoring and Evaluation Plan? What were the reasons that key performance targets were or were not met?
2.1.2. To what extent did the program strengthen institutional capacity for disaster and climate change management among BPBDs and C/DRRCs? What have BPBDs and C/DRRCs achieved as a result of increased capacity?
2.1.3. To what extent was the program able to reduce exposure and vulnerability to disaster by increasing technical knowledge of, adoption, and income from adaptive livelihoods?
2.1.4. To what extent did the program improve disaster preparedness practices among targeted communities through knowledge and education? What were the major factors influencing the adoption of improved disaster preparedness practices?
2.1.5. To what extent did program interventions impact women beneficiaries in addressing gender-specific needs to climate change adaptation (CCA)? How did these interventions help women respond to shocks and stresses to mitigate their risks?
2.3. Relevance of the Design and Program Coverage
2.3.1. Was the program design relevant to the targeted beneficiaries? What are the key determining factors that make this program design relevant?
2.3.2. What components of the project design had a positive impact on DRR and CCA, and what components had little/no impact or could have been improved? Components include: community mobilization, constitution of DRM committees, approach to adaptive livelihoods, linking groups with other stakeholders, partner performance, and technical service provision to groups
2.3.3. What is the current program coverage for improving CCA and DRR knowledge and practices? Are there any significant differences across geographic areas?
2.3.4 . Is the program design replicable and scalable? If so, what factors will determine where this program can or cannot be replicated and scaled?
2.4. Sustainability
2.4.1. Are the groups formed during the program (DRR groups, Livelihood groups, and organic farming groups) functioning independently?
2.4.2. Have the above mentioned groups formed linkages to the available institutional and financial resources? If linkages have been formed with available resources, what outcomes have been generated, and are the linkages sustainable?
2.4.3. Have beneficiaries created any innovations based on the adaptive livelihood trainings that could make them more sustainable?
2.4.4. Did the adoption of adaptive livelihoods have any negative impacts that would affect their sustainability?
2.4.5. What is the current policy environment influencing the likelihood of sustainability for these groups?
2.4.6. Has the program created any “champions” who can continue to lead the process of developing resilient communities?
Through the process of responding to the research questions, the external evaluation consultant is expected to identify lessons learned, challenges and unexpected results to provide concrete recommendations for improved future program design and implementation. The consultant will also document best practices to provide recommendations for organizational learning.
III. Evaluation Design: Sample, Data Sources and Research Methods
Sample Population
he evaluation consultant is expected to propose an adequate sample that will respond to the proposed research questions. The sample should take into consideration all of the following stakeholders:
- Regional Body for Disaster Management (BPBD)
- Agriculture Department
- Food Security and Extension Worker Department
- Education Department
- Religious Department
- Village Committee
- Representative of economic group
- Representative of organic farming group
- Representative of village official and village leader
- Representative of District official
- Representative of teacher worked on School based disaster management
Both primary and secondary data sources should be considered in the development of the program evaluation design. Primary data sources should consider the entire CADRE population and appropriate respondents/key informants. Secondary data sources include program source documents, e.g. CADRE contract agreement, work plans, reports (baseline, mid-term, annual, etc.), policy documents, etc.
Research Methods
A mixed-methods approach for collecting both quantitative and qualitative data is preferred. Data should be representative of the program intervention and can include document reviews and synthesis, focus groups, key informant interviews, observations, surveys, among other kinds of research methods.
Quantitative data should be managed using appropriate data management technology and analyzed using statistical software (e.g. R, SPSS, STATA, Epi-Info, etc.). Quantitative and qualitative data should be presented using the appropriate data visualizations to effectively represent key results.
IV. Application Instructions
Applicants must submit a cover letter, CV, 3 professional references, and a USAID Biodata 1420 form to be considered for the consultancy. Submit your application to Arief Firdaus, mafirdaus@pci.or.id, with a copy to hmaad@pci.or.id with “Application-CADRE Final Consultancy” in the subject line by 21 Nov 2014.
V. Requirements for the External Evaluation Consultant(s):
The person or firm conducting the evaluation should meet the following criteria:
- Minimum of a master’s degree in social sciences, climate change, international development, or related field
- Extensive experience in conducting evaluations using both quantitative and qualitative techniques, with a minimum of five years of experience in this area.
- Experience in evaluation of climate change, disaster risk management, alternative livelihoods, economic development or a related area
- Professional experience conducting evaluations in rural Indonesia
- Experience in evaluating US Government-funded programs, particularly USAID
- Knowledge/familiarity with Indonesia’s national legislation relating to climate change and disaster risk management.
- Proven capacity to deliver high quality results within the proposed timeframe.
For those who are qualified and interested in this consultancy work, please submit your technical proposal along with budget and CV/resume to: Arief Firdaus, mafirdaus@pci.or.id with a copy to hmaad@pci.or.id and Indonesia CADRE Final Evaluation as the subject of your email.
Latest date for proposals will be on 28 Nov 2014. Only shortlisted consultant will be contacted for recruitment
