1. Background

Health Poverty Action (HPA) has been working in Burma’s ceasefire areas bordering with China since 1994, primary health has been the focus of HPA’s work. These areas had undergone decades of conflicts with the central government and just reached ceasefire agreements separately with them in around 1989. The availability of health services was extremely limited when HPA began working here, and those that were available were fragile, fragmented and with very limited capacity. Due to factors, such as political sensitivities and remote border locations, only a handful of international NGOs are able to access these areas. After 20 years of work, HPA has extended its programme to nearly all townships on the Burma-China border which were under the control of the ceasefire groups or Union government.

HPA has registered at Myanmar Ministry of Home Affairs in 2012 and signed MoU with Ministry of Health and Sports in 2013. Currently, HPA is receiving fund from UKAID, DFID Burma, Global Fund (Malaria & HIV grant), 3 Millennium Development Goal Fund and UNICEF to provide integrated packaged primary health care to population in 21 hard to reach townships along Myanmar-China border. The service provided includes: Maternal and new-born care, expanded programme on immunization, reproductive health, family planning, water & sanitation, common disease treatment, health education, communicable disease prevention and treatment, community capacity building and health system strengthening.

2. Purpose of the Evaluation

Even though HPA has multiple funding sources to invest on health in the same geographic area. HPA does not run the programmes vertically funded by different sources, but integrate all the available resources and flow into local/ethnic health system to response to the health need in a complimentary approach. The purpose of the proposed evaluation is to evaluate overall impact of organization’s health programme. However, the evaluator need to differentiate the contribution of each funding source, which are DFID Burma and 3MDG Fund, with the support of HPA’s staff.

3. Effects of the Project

Changes

What evidence is there that the outputs have led to the fulfilment of the project purpose?

With reference to the logframe indicators and, what progress has been made towards the project purpose?

Are there any unplanned consequences of the project, whether positive or negative?

Target

To what extent are traditionally disadvantaged groups benefiting from the project, for example considering

     gender, ethnicity, socio-economic group or HIV status?

Sustainability

Considering the financial, institutional and socio-cultural aspects of sustainability:

Given current relationships and implementation progress, what is the likelihood of the project having a sustainable impact in the longer-term?

What changes are recommended to facilitate sustainability?

Project Implementation

Planned vs Actual

To what extent has the project delivered the activities outlined in the logframe, and what are the reasons for any over or under performance?

To what extent are the project activities reaching the target groups, and what are the reasons for this result?

Is the project delivering other activities not included in the logframe? If so, what are the reasons for this?

With reference to the logframe indicators, and the contribution of the activities, what progress has been made towards the planned project outputs? What are the main reasons for any under or over-performing?

Is the project likely to achieve its planned outputs by the end? If not, what changes should be made to implementation? How does this compare to previous phases?

Please note: It is needed to separate documents to discuss Planned vs Actual against DFID PHC and 3MDG logframe separately.

Processes

Assess the strength of the partnerships between HPA and local partners and MoHS/SPHDs (state public health departments). How could the relationships be made more effective in the remainder of the project?

Regarding the alignment with the Myanmar health system, what has been done and what has been the impact (both negative and positive aspects) on the programme itself and on the health system. What recommendations can be made on how the project should be prioritized in order to improve the quality of service and scale of coverage?

Have other partnerships, other than those originally planned, been developed?

Is the project organised in a way that ensures optimal (i) participation and (ii) ownership of (a) the local partners and (b) the local/target population?

To what extent have the beneficiaries been involved in the planning, design, implementation, monitoring and evaluation of the project? And how is the project managing the role of volunteers and health facilities?

What are the strengths and weaknesses of the approaches adopted to implement the project, on an output-by-output basis?

How might the project’s approaches be improved?

To what extent have (i) gender, (ii) disability and (iii) HIV issues been considered in implementation?

What steps have been taken to ensure donor and HPA visibility?

How is HPA managing the integration of its programmes? What’s the approach and practise?

Management

Is the staffing and management structure appropriate to fulfil the project purpose and outputs?

Is implementation systematically monitored, and the data used to inform decision-making?

Are the logframe indicators appropriate for measuring the project outputs and purpose? If not, and in discussion with project staff, what indicators would be more appropriate?

Constraints

Are the assumptions and external constraints originally identified in the project design still valid? What significant new opportunities or constraints have arisen during the project?

To what extent is government or local policy supporting or constraining project implementation?

What internal factors help or hinder the project?

What is the project doing to mitigate against these external and internal constraints?

Wider Potential

Lessons to be learnt

What approaches taken by the project can be regarded as innovative or as appropriate adaptations of good practice? (particular reference should be made to known cultural and socio-political appropriateness of the approaches)

What lessons learned could be used beyond the project area: (i) within Burma); (ii) within HPA (e.g. management, administration);

Future

At this stage of the project, what would be an appropriate direction for HPA’s future work in this project area based on local needs or national or regional concerns?

How could DFID PHC and 3MDG programme merge as one programme in the future?

4. Methodology of the Evaluation

The evaluation will make use of various methods including desk review of project documents and records, field visits, observation, interview of key informants including HPA field staff, local officials, local clinic managers/staff, TBAs and beneficiaries and focus group discussion as necessary. The use of appropriate participatory approaches is essential. Methods to be used could include:

Review of project documents, these will be sent one month in advance to the review team;

Review of project financial and administrative records and procedures;

Interviews with the project partner;

Interviews with other project stakeholders

Interviews with the project staff;

Discussions and consultation with health units at all levels;

Discussions with donors

Review of health facility records

Review of training records

Focus Group Discussions with the beneficiary community members

Observation of project activities and “village life”

Photographs

5. Background Reading

Project proposal and logical framework

Project budgets

Annual progress report

Six monthly reports

2017 Endline Survey Report

National Health Plan 2017-2021

6. Profile of the Evaluation Team

The evaluator team will ideally have the following skills/experience:

Public health specialist, preferably specialized in primary health care and/or health management.

Minimum five years experience in managing health projects/programme.

Experience in evaluating health projects.

Able to deliver report in English.

Knowledge of the historical and political situation of the ceasefire areas of Myanmar is desirable.

The evaluation team will consist of one external consultant who will be the team lead, responsible for:

coordinating the allocation of work

facilitating an in-country debrief

writing the draft and final reports, with contributions from the other review team members

In addition two HPA staff members will participate in the evaluation, and interpreters (where required). Background information will be provided through coordination with local level government representatives.

7. Timing

The evaluation will be carried out during December 2017. The draft report will be submitted to HPA no later than 20 days after the field visits, and the final report will be available no later than 5 days after receiving clarifications from HPA.

The detailed itinerary will be finalised after the discussion among the reviewer, field office and EAPO.

EVALUATION (20 days)

Preparation: 4 days

The reviewers will review the literature and prepare tools.

Field work: approximately 12 days (including 1 day for debrief)

Interviews of some key informants will be conducted before reaching the project site. This includes 1 day for draft report writing.

Report writing: 4 days (including to incorporate feedback)

Evaluation Teams:

Lead Consultant

One interpreter (if required)

All collected data will be integrated and analysed. The draft report will be circulated among all key stakeholders involved and their feedback will be used to correct factual errors and address requests for clarification. The reviewers will make the second draft report. Further work may be needed if errors or clarification requests have not been addressed.

8. Report

The report will include the following:

Overview of programme achievements against the approved logframe.

Analysis of why or why not the programme achieved the expected objectives at outcome and output levels.

Comment on the impact and sustainability of the programme.

Summary of lessons learnt.

List of recommendations as well as the rationale for providing these recommendations.

Format

The final report should be a maximum 20 pages, excluding annexes, and should be written in English. It should contain an executive summary of a maximum 2 pages. The report should follow the following format:

Title page

Short description of reviewers

Acronym list

Executive Summary

Introduction/context

Objectives

Methods

Constraints

Case Studies

Findings

Conclusions

Recommendations

Annexes

The report may include quotes, photos, graphs etc.

9. Logistics Arrangements

The visits and meetings will be arranged by HPA.  Please note that visitors must be in compliance with Health Poverty Action’s Child Protection Policy.

Contact

MR. Cai Tao

Email: caitao.km@qq.com