Terms of Reference for an External Evaluator
These Terms of Reference (hereafter: TOR) are intended to provide guidance and reference to Caritas Germany and Karina KAS and PPRBM Solo in arranging the evaluation of their project. It is also intended to provide guidance to all third parties interested in facilitating the external evaluation.
Based on this TOR,
interested parties are asked to send in their proposal and bidding to
facilitate the evaluation.
Based on criteria as outlined in this TOR an
evaluator will be selected.
Project: Second phase continuation and expansion of BMZ project
P.416-023/2008 BMZ PN: 200926006
“Community Based Rehabilitation (CBR) Aid for Disabled
People in Central Java, Indonesia”
Implementing Partners: Karina KAS and PPRBM Solo
Project Period: January 2013- December 2015
Caritas Germany is
a part of the German Catholic Church, and as such has received the mandate to
implement projects that are in line with the social responsibility of the
Church, both in Germany and international. The main focus of our work is
towards emergency response, helping the elderly and the ill, children and
disabled people. In Indonesia, Caritas Germany has been supporting local
partners since the Tsunami of December 20014.
More information
can be found on www.caritas-germany.org.
Karina KAS –
Karitas Indonesia, the Caritas organization of the Archdiocese of Semarang -
was established in May 26th, 2006 to give Emergency Response of Yogyakarta
Earthquake 2006. Since then, Karina KAS has been mandated by the Archbishop of
Semarang to be the institution that represents the social face of the Church. Based on the needs after a disaster and guided by the WHO-CBR matrix, a CBR program was started by Karina KAS in 2007. This was further developed into a comprehensive CBR project addressing Health, Education, Livelihood, Social and
Empowerment according to the WHO CBR matrix in 3 Districts as the first cycle of a 3
year BMZ funded project (2009-2012), followed by the
second cycle of a 3 year BMZ funded project (2013-2015). More information can be found
on
www.karinakas.org.
PPRBM Solo was
established in 1978 by Yayasan Pembinaan Anak Cacat (YPAC) or the Indonesian
Society for the Care of Disabled Children. PPRBM Solo was formed after YPAC
realized that institutional based services are not enough to serve the majority
of children with disabilities, particularly those living in rural areas,
therefore they started to develop the CBR concept as an approach to disability
issues in many rural areas. PPRBM Solo has collaborated since 2010 with Caritas
Germany developing Disability Advocacy Teams (DATs) at District level where the
Indonesian government has autonomy. Besides that, the program worked with Self
Help Groups (SHGs) to develop small businesses or enter the general workforce.
As an expansion of the CBR BMZ program of the 1st cycle, their concept of
working with the Indonesian government at District level through the Disability
Advocacy Teams to implement the rights of PWDs, was fitting well into the
continuation of the CBR program implemented by Karina KAS. As the UNCRPD was
ratified by the Indonesian government, the strategy of PPRBM Solo tried to
enhance the implementation of the UNCRPD by the Indonesian government. PPRBM
Solo has become an independent organization since 2014. More information
can be found on: www.pprbm-solo.org.
2. Background of the
project
In Indonesia
Caritas Germany is supporting projects with a wide variety of partners
throughout Indonesia for emergency response, in drugs abuse, HIV&AIDS and
working with persons with disabilities (hereafter PWDs). For the support for
PWDs Caritas Germany tries to implement and promote the strategy of Community Based Rehabilitation (hereafter: CBR) using the WHO CBR guidelines and the CBR-matrix of the World
Health Organization (hereafter: WHO).
One of Caritas
Germany’s partners for the CBR program is the Caritas organization of the
Archdiocese of Semarang (hereafter: Karina KAS) with their office in
Yogyakarta. After the
earthquake in Yogyakarta and Central Java of the 27th of May 2006
and when the emergency period was finished, a comprehensive CBR team was formed
in January 2007, using the WHO CBR Matrix. The team consisted of a health,
education, livelihood, social and empowerment division addressing disability issues. The program was expanded through the Catholic network
to Bantul in May 2007. In 2009 the program was duplicated to
Sukoharjo. This program was funded by
the German Ministry of Economic and Development Cooperation (BMZ) from
2010-2012. In 2013 Karina KAS began to implement the second cycle of this BMZ
funded program together with PPRBM Solo.
3. Project Objectives
The overall aim of
this project is: “The rights of PWDs are fulfilled within an inclusive society”
The objectives of
the project are:
1. To
enhance access to quality Public health Centers and facilities for PWDs.
2. To
enhance access to quality education services and facilities for PWDs.
3. To
increase capacity and access of PWDs and their family towards sustainable
livelihood.
4. To reduce
self and social stigma of PWDs in society.
5. To
empower CBR stakeholders in particular PWDs and family to actively advocate for
the rights of PWDs in development.
6. To increase the capacity of implementing partners in project management and implementation.
6. To increase the capacity of implementing partners in project management and implementation.
This project
consists of 2 components (A and B) implemented by Karina KAS and PPRBM Solo:
Karina KAS:
The program is
implemented by Karina KAS at the community level in the regions of Bantul (DI
Yogyakarta), Klaten and Sukoharjo (Central Java) , integrating a network at
district level, provincial and national level. Sukoharjo was exited mid 2014 as
all objectives were achieved and replaced by Gunung Kidul and started mid 2014.
PPRBM Solo:
The program is
implemented by PPRBM Solo in 7 districts in Central Java: Sragen, Boyolali,
Karanganyar, Klaten, Wonogiri and Grobogan districts, and the city of Surakarta
(Solo). These districts are linked to province level (Central Java) and
national level.
A:
This component is
at grassroots level, direct on the ground addressing health, education and
economy, implemented in 12 villages and 3 districts. Direct beneficiaries are
800 People with Disabilities in 12 villages, with indirect beneficiaries, their
families 3200 people. The program focuses on 8 Community Health Centers at sub-district
level and 98 integrated health service posts at village and 115 health cadres
and midwives associated to this health system. 328 People with Disabilities
will be identified through health cadres and will receive direct support in the
form of assistive devices. 3 Cooperatives at district level will be
strengthened and further developed with a total of 300 members. There are 19
Self Help Groups with a total of 570 members. One inclusive primary school and
12 playgroups with 20 mainstream teachers for inclusive education and special
education teachers will be trained to work in inclusive education. 12
Karina KAS staff members and 12 PPRBM staff members will get capacity building
training.
B:
This component is
advocacy oriented, working through 7 Disability Advocacy Teams and 3 Community
Based Rehabilitation boards at district level, having an impact on the more
general population of 8,203,334 in 9 districts and on 35,833,077 in (2)
provinces with joint collaboration in 1 district (Klaten). As the general
population in these 2 provinces is 35,833,077 (based on the latest census of
2010) and using the general estimation of the World Health Organization (WHO)
that 15% of the general population is disabled, an estimated 5,374,962 People
with Disabilities live in these 2 provinces.
Reason and planning context of the Evaluation
Currently Caritas
Germany, Karina KAS and PPRBM Solo are running towards the end of the three (3)
year Phase II running from 2013 – 2015 and are planning for Phase III that
should run from 2016 – 2018. For this a new program and proposal should be
prepared by mid-2015, based on best practices and lessons learned during the
current Phase II. In order to plan for Phase III in a correct way, Caritas
Germany is looking for an external evaluator to evaluate the current CBR program
implemented in Indonesia through Caritas Germany’s partners.
4. Evaluation Criteria
Caritas Germany
together with Karina KAS and PPRBM Solo are seeking an in depth evaluation of
the project achievements under the following 5 criteria and make specific
recommendations to improve each criteria.
1. Relevance: Are we doing the right thing?
The extent to which the intervention’s objectives are in line with the
needs of the target groups, the policies of inclusion at village, sub-district,
district, province and national level in Indonesia and its partner institutions
as well as with global development goals such as the post MDGs, UNCRPD and CBR
guidelines.
A. The relevance of the
objectives and strategy;
• Are the project objectives
and strategy still relevant?
• Does it still correspond
to priority needs of the target populations?
• Are the data it is built
upon sufficient and updated?
• Is
the project regularly updating the changes in the policies and stakeholders to maintain
its relevance up to the mark? How?
FOCUS of the Recommendation:
• Which areas of the project
are more/less relevant to develop? How?
2. Effectiveness:
Are we attaining the objectives of the intervention?
A measure of the extent to which an aid activity attains its (direct)
objectives (target-performance comparison)
A. Effectiveness of results as
compared to specific objectives
• Does the project attain
fully all of its objectives?
• Is
the project management system operational and sufficient with respect to the
expected results?
• Is its monitoring system
sufficient applied?
FOCUS of the Recommendation:
• Is there any need to
reinforce the monitoring & planning system? How?
3. Overall
impact: Is the intervention conducive to reaching overall goals?
In the context of overall impact the following must be considered: Whether
and to what extent does the intervention contribute to reaching overall
development policy objectives? In addition: Whether and what type of positive
and negative changes have occurred?
A. IMPACTS
• What are the main changes
in the target group living conditions?
• Is the project responsible
of positive changes at Indonesian policy level?
• Is
the impact of the program recognized among the different stakeholders?
• Is the project equipped
and organized to measure its impact?
FOCUS of the Recommendation:
• Is
there a possibility of a better measurement of the impact (economical, social
and political effects)?
• How could the project gain
in impact?
• Suggest impact measurement
indicators for phase III.
4. Sustainability: Is positive impact permanent?
Sustainability measures whether the benefits of an activity are likely to
continue after donor funding has been withdrawn.
(from: German Federal Ministry for
Economic Cooperation and Development
(BMZ)
– Evaluierung der
Entwicklungszusammenarbeit)
A. Sustainability of the project
• Are
the attained results permanent, long-term gains?
• May
the project be replicated? Extended? How and where?
• The
Phasing out strategy is it clear and feasible?
FOCUS of the Recommendation:
• The recommendation will focus on
examining the possibility and potential mechanisms for sustainability specifically
in health, education, livelihood and social at community level
• Is
there a need to review the phasing out strategy? What key points should be
stressed?
• How
could the action be financially and operationally sustainable on the long term?
5. Coherence:
The need to
assess developmental and government policies in the inclusion of PWDs,
A. External and Internal coherency:
• Are they in line with the local and
national policies?
• Are the activities conducted in
coordination/cooperation with other local actors?
• Do the chosen activities permit to
achieve the wanted results?
FOCUS of the
Recommendation:
• Is the program in line with the local and national policies in inclusion of PWDs? Should they use different approaches or strategies?
• Is the program in line with the local and national policies in inclusion of PWDs? Should they use different approaches or strategies?
5. Evaluation Methodology
The eventual
methodology depends on the proposal of the external evaluator and will be
defined during the preparation of the evaluation. However, Caritas Germany
suggests that the methodology analyses both quantitative and qualitative data
and provides concise input for phase III (2016-2018).
6. Expected Outputs
§
An evaluation report in English with an
executive summary in Bahasa Indonesia and English. This report should be
presented in five (5) hard copies, and a soft copy both in MS-Word and PDF
format.
§
A presentation for Caritas Germany and our
partners working in CBR in April 2015, to take place in Yogyakarta.
7. Evaluator Profile
The third party evaluator should not in any
way be related or dependent on either or both Caritas Germany, Karina KAS or
PPRBM Solo, or be involved in any way in the project being evaluated. Besides
this the evaluator should have:
§ Good knowledge, understanding of and experience in evaluating external funded projects for at least five (5) years;§ Good knowledge, understanding of and experience in Community Based Rehabilitation;§ Good understanding of verbal and written Indonesian;§ Good understanding of verbal and written English;§ Caritas Germany is actively trying to be an inclusive organization, and thus on equal grounds will give priority to offerings involving people with a disability.
8. Bidding and Evaluation Plan
Based on these TOR, Caritas Germany, Karina
KAS and PPRBM Solo request any party/person(s) who are interested and able to
facilitate this evaluation, to submit a bidding to facilitate this evaluation.
The bidding should at least include:
§ An offering in IDR to facilitate this evaluation;§ A detailed work plan and methodology proposed for facilitating the evaluation (maximal 5 pages);§ A company profile and/or CV of the evaluator(s) showing the experience in facilitating an external evaluation and knowledge of the subject at hand;
This information should be send via email no later than 22 February
2015 to caritas.germany@gmail.com.
After selection and approval of the evaluator, the evaluator has to send a
signed hard copy of these documents to Caritas Germany
