Evaluation of the ‘Emergency Nutrition Assistance for Drought Affected Vulnerable Communities of Turkana county, Northern Kenya project funded by USAID’s Office of U.S. Foreign Disaster Assistance (USAID/OFDA)
Terms of Reference (ToR)
1. Introduction and background information:
Merlin is an international health charity, saving lives in the world’s toughest places. Merlin responds with healthcare, when people are overwhelmed by natural disasters, conflict or disease and in need of immediate help. Merlin focuses on treating those who are beyond the reach of existing health services and stays on to assist recovery after the immediate crisis.
Merlin supports health workers and strengthens existing health services while building resilience by helping those at risk of future health disasters to be better prepared.
Merlin has been operational in Kenya since 1998 and is currently working in two regions of the country: the Western Highlands and North Western Kenya, including Turkana.
Turkana County covers about 77,000 square kilometres of the country’s arid North Western region and it borders Uganda, Sudan and Ethiopia to the West, North West and North East respectively.
The county is classified as arid and semi-arid lands of Kenya (ASAL) and is historically drought prone area that experiences frequent, successive and prolonged dry periods. It has two rainy seasons, the long rains (April–June) and the short rains (October–December) but rainfall is often unreliable.
The 2011 drought that affected the Horn of Africa region exposed many pastoralist communities, including those living in Turkana County to severe food insecurity which eventually led to high rates of acute malnutrition amongst children under five years and pregnant and lactating women [Turkana nutrition survey – May 2011].
Turkana North East reported the highest GAM rates of 37% and SAM rates of 9.4%, with the rest of the other three districts ranging between 24.4% to 33.5% GAM, and 4.5% to 6.8% SAM.
This then prompted a massive scale up of life saving interventions in July 2011 by Merlin and other partners which eventually resulted in a significant reduction in the acute malnutrition rates in the county.
According to the latest survey (July 2012), Turkana South and North West districts recorded GAM rates of 11.6% and 14.3% respectively, which depict critical levels of acute malnutrition but are lower than the WHO emergency threshold of 15%, while Turkana South and North East districts recorded GAM rates of 17.1% and 15.3% respectively, which are slightly higher than the WHO emergency threshold. Apart from Turkana South which recorded SAM of 4.2%, all the other three districts registered SAM figures of <4 nbsp="nbsp" span="span">
Overally, these latest rates are significantly lower than the survey results recorded during the same season last year (May 2011) [Turkana nutrition survey results – May 2011 and July 2012] .
2. Project Objectives and Expected Results
The overall goal of the OFDA project was to contribute to reduction in levels of morbidity and mortality amongst drought-affected vulnerable communities in Turkana County, particularly under-five children (U5) and pregnant and lactating women (PLWs).
The approach being taken to achieve this goal is to provide health facilities and outreach-based integrated nutrition assistance services targeted towards four key areas :
1) management of moderate acute malnutrition (MAM) and severe acute malnutrition (SAM),
2) improvement of infant and young child feeding (IYCF) practices,
3) nutrition education and
4) behaviour change communication. All these are aimed at supporting the Ministry of Public Health and Sanitation in implementing the high-impact nutrition interventions (HINI).
The project is also based upon the partnership framework between the MoH, UNICEF and partners to support the delivery of essential nutrition services in Kenya.
This partnership framework envisages that general principles of support to MoH should include:-
- Full integration of a comprehensive package of high impact nutrition interventions.
- Results based orientation to programming.
- Full commitment to coordination.
- Strategic development of capacity at all levels of service delivery to enable the MoH function well with minimal or no support from partners.
- Direct involvement in service delivery only in exceptional cases
- Focus on mentoring, supportive supervision and on-the-job training.
3. Overall objective of the Evaluation:
As the ‘EMERGENCY NUTRITION ASSISTANCE FOR DROUGHT AFFECTED VULNERABLE COMMUNITIES OF TURKANA COUNTY’ grant is coming to a close, Merlin wishes to undertake an external evaluation of the project – focusing on achievements, successes, lessons learnt and best practices.
These will be assessed against the OECD-DAC criteria that are most relevant to the purpose of this evaluation.
4. Specific Objectives:
i. Outputs and outcomes: Outputs and outcomes generated by the program in relation to the stated goal, objectives and desired results. In particular, the evaluation will assess the effects of the program activities on the targeted beneficiaries in OTP, SFP and community members reached with the various services.
ii. Relevance: whether the program interventions met needs of the beneficiaries; the appropriateness of results in relation to the needs of the communities, national policies and priorities.
iii. Effectiveness: In particular assess the extent to which program interventions achieved the desired outcomes, factoring in issues of program management including decision making processes, risk management, institutional arrangements and partnerships with MOH and other agencies and their effects on the program results.
iv. Efficiency: the relationship between the quantity, quality, and timeliness of program inputs, including personnel, training, office equipment. In addition, determine the quantity, quality, and timeliness of the outputs generated and whether the resources were spent as economically as possible.
v. Sustainability: assess the readiness of partner (MOH) and other stakeholders to sustain program interventions, in particular assess the infrastructure and systems of partners, resources available to sustain the activities and services, collaborative links and referral networks with other service providers, and the level of community ownership.
vi. Program Improvement areas: capture key successes, best practices lessons learnt, implementation challenges, constraints, strengths and weaknesses.
vii. Determine the extent of complementarity of this project with other related health and nutrition projects currently supported by UNICEF and DEC in Turkana County.
viii. Role of beneficiary consultation in programme design and implementation, and areas for improvement.
This will be assessed against accountability commitments in Merlin’s Accountability Framework.
5. Methodology
The following methodologies shall be used for the evaluation:
Evaluation Design
Both quantitative and qualitative methods may be used for the study.
The qualitative method will for example employ focus group discussions with target groups, employ interviewer administered structured questionnaires to stakeholders - government, beneficiaries, MoH, WFP and UNICEF, as well as in-depth interview of key informants and document review, both primary and secondary.
The quantitative method will review existing data related to the outcome indicators supplied by Merlin/MoH, and verify sample of same in the field.
The consultant will use information both from primary and secondary sources which are directly related to the project.
For triangulation purposes, the consultant is expected to refer to documents from Merlin/MoH reports and national and sub-national statistics, research findings relevant to this project and donor guidance and associated documents.
These sources will be used to validate the results and triangulate successes and failures for lessons documentation.
Key evaluation questions
1.1. How far has the implementation and delivery of the project activities, especially the choice of beneficiaries and organisational modalities, facilitated the achievement of specific objectives?
1.2. To what extent has the project addressed the emergency nutrition needs identified in the target communities as reflected in the May 2011 nutrition survey?
1.3. What is the level of integration of health and nutrition activities in the MOH facilities in areas of Merlin operation and what lessons can be derived from current implementation approach?
1.4. What is the level of sustainability for the project results? What linkages have been created and or exploited by the project to sustainably address malnutrition among the target communities?
1.5. What are the costs and effectiveness of implementing the health and nutrition activities in Turkana? Are the project implementation approaches cost effective in relation to practical realities on the ground?
1.6. What lessons have been learnt through the implementation of this project?
Study Area
The evaluation will be conducted in locations determined by the evaluator(s) in consultation with Merlin, but has to be within Merlin’s geographical area of operation.
Data collection and analysis
- The evaluation should be participatory and focus on obtaining both qualitative and quantitative information.
- Quantitative data obtained from Merlin on key indicators to triangulate with baseline data.
- Qualitative data obtained from FGDs, key informant interviews and structures questionnaires.
- Project document review, individual interview of project staff, government, UNICEF, WFP and other partners.
6. Responsibilities and lines of communication
Merlin’s programme cycle management defines the members of the organisation and external stakeholders that are accountable, responsible, consulted and informed at each stage of an evaluation.
The persons named below will ensure that a suitable and quality evaluation is conducted within the parameters of Merlin’s evaluation policy
Table 3: Responsibilities and lines of communication
| Accountable | Responsible | Consulted | Informed |
Evaluation Design | Country Director (CD) | Country Health Director (CHD) | Project Coordinator (PC), Senior Project Officer, health and nutrition (SPO-HN), Nutrition Coordinator (NutCo) Donor, Beneficiaries MOH. | Other partners Community Other staff |
Evaluation Management (Implementation) | CD | CHD, PC | PC, SPO-HN, NutCo, MOH Community | Staff other partners Beneficiaries Community |
Reporting / Dissemination | CD | CHD | PC , SPO-HN, NutCo, Donor Beneficiaries, MOH | Staff, other partners |
Evaluation Learning | CD | CHD | PC , SPO-HN, NutCo, MOH Beneficiaries Other Staff | Beneficiaries Community |
7. Responsibility of the consultant/deliverables
The Consultant is expected to undertake the following tasks:
- Design and develop appropriate tools/mechanisms that will enable the analysis of the supplied information.
- Analyse collected information and write up a report detailing findings using Merlin’s evaluation policy and templates.
- Submit draft report for review
- Present final report to Merlin.
- Submit a final hardcopy report as well as soft copies in word and PDF formats
- Preparation of a PowerPoint of the evaluation methodology and key findings, for further dissemination, and a delivery of that presentation to key project staff.
- The evaluation process must be impartial, independent and must provide information that is credible and useful, enabling the incorporation of lessons learnt, and identify the strengths, weaknesses, opportunities and threats to inform future decision-making process within Merlin, MoH and donors.
8. Key outputs of the evaluation are
- A report completed on Merlin’s Evaluation Report template (final hardcopy report as well as soft copies in word and PDF formats);
- A PowerPoint of the evaluation methodology and key findings;
- A focus group discussion held with the Country Management Team and Regional Desk (either in-country or from Head Office after the visit) prior to drafting the evaluation report to discuss preliminary findings and recommendations;
- A report completed on Merlin’s Key Learning document template, and; an Action Plan that details activities and timescales to meet recommendations (completed by the country team and the desk).
9. Required Qualification and Experience
The consultant should have the following qualifications and experience:
- Relevant academic and professional background in humanitarian interventions in complex environments.
- Extensive practical experience in conducting evaluations of humanitarian interventions in the ASAL regions of Kenya with preference given to significant emergency nutrition programming.
- Technical research skills, including qualitative and quantitative approaches.
- Fluency in both written and spoken English and Kiswahili.
- Knowledge and experience of working in Turkana is preferred.
10. Proposal details and submission
The deadline for submission of bid (both technical and financial proposal) is before close of business on September 7, 2012.
Soft copies of technical and financial proposals including CV outlining relevant consultancy work undertaken (including references) should be submitted, including the title of this ToR to recruitment@merlin-kenya.org.
The evaluation criteria are based on technical and financial responsiveness which also includes a valid licence and tax registration.
The key technical evaluation criteria are:
- Known reliability in delivery of timely and quality services
- Relevant field/country experience
- Relevant sector specific technical experience and qualifications
- Relevant evaluation experience
- Presentation and writing skills evident from the proposal
- Demonstrated understanding of the work proposed in the TOR
- Clarification of methodology
- Cost/budget
11. Ownership of the deliverables
Merlin shall have full ownership of the report, research, working papers and other results of the Consultant’s work under this assignment.
The Consultant shall not publish nor otherwise use any portion of the report without the written authorization of Merlin.
The consultant shall waive any copyright to the work produced as a result of the contract.
12. Timeline
In consultation with project staff (Merlin), the consultant shall design the review work plan and develop the evaluation instruments including appropriate questionnaires. The review is expected to take two weeks and additional six paid days have been catered for report writing.
The study is expected to apply participatory approaches along with quantitative and qualitative methodologies. It is expected that the evaluation will begin in September 2012 with a final report due no later than end of October 2012.
The MoH staff at the county level, Merlin staff, community health workers who are directly involved in the project implementation, and the community members (children, women, and men) will be targeted for obtaining qualitative data for the study.
Quantitative information will be gathered from existing MOH/Merlin data.
13. Format for the evaluation report:
The final report shall at least consist of the following sections:
a) Executive summary
b) Project description
c) Context of the evaluation
d) Evaluation methodology
e) Findings
f) Lessons learnt, best practices, and recommended actions for the review of primary users
g) Annex (es) as necessary
14. Logistics / Transport / Accommodation
Merlin will facilitate meetings with stakeholders and support the consultant in conducting interviews and other activities. Merlin will also take care of the consultant’s accommodation while in the field.
