PROJECT EXTERNAL FINAL EVALUATION
来源:Handicap International - Humanity & Inclusion
2023-06-15
188

1. Background

Humanity & Inclusion (HI) is an independent and impartial aid organization working in situations of poverty and exclusion, conflict and disaster. We work alongside people with disabilities and individuals experiencing extreme hardship, taking action and bearing witness in order to respond to their essential needs, improve their living conditions and promote respect for their dignity and fundamental rights.

HI runs more than 400 projects in 60 countries in South America, Africa, the Middle East and Asia. In China, HI has been working with government and civil society organizations since 1998, promoting the socio-economic inclusion of people with disabilities.

Promoting the dignity and fundamental freedoms of people with mental health disabilities project (MH) is a 36-month project (01 Nov 2020-31 Oct 2023) funded by the Bread for the World (BftW) and implemented by HI in Hunan, Sichuan, Chongqing, Yunnan and Beijing provinces with a total budget of EURO 500,000.00.

The overall goal of the project is that the dignity and fundamental rights of people with mental health disabilities is protected in China. The project will contribute to the realization of this goal by supporting CSO contribution to the development, documentation, and dissemination of models of good practice in line with the UNCRDP and SDG goals. This specific objective will be realized through the delivery of three desired outcomes:

1.      Outcome One: Civil society organizations with a mission to support people with mental health disabilities have an improved understanding of relevant international and domestic laws and policies. 

2.      Outcome Two: Civil society organizations have an improved capacity to support delivery of community based mental health rehabilitation for people with mental health disabilities.

3.      Outcome Three: Civil society organizations contribute to the definition of local and national policies related to community based mental health rehabilitation.

Expected results and indicators

Expected Result One:

Indicator 1.1 At least 75% of 28 targeted CSO staff and 30 targeted Heart Wing and Dong Qing support group leaders will have an increased understanding of international and domestic legal frameworks and good practices related to the design and delivery of community based mental health rehabilitation services (at least 50% of targeted CSO staff and Heart Wing and Dong Qing support group leaders will be women).

Expected Result Two: 

Indicator 2.1 The overall scoring of 2 targeted CSOs is improved in regard to governance, management and operational capacity (measured with standardized organizational capacity development self-assessment tools).

Indicator 2.2 At least 75% of the 100 people with mental health problems (with at least 50% of whom will be women) who participate in the services provided by the targeted CSOs report an increased level of satisfaction with the quality of service provision by the end of the project.

Expected Result Three:

Indicator 3.1 Until the end of the project, at least 5 policy proposals were formally submitted to targeted authorities by CSO or Heart Wing and Dong Qing support groups supported by the project, providing recommendations for the local implementation of the National Mental Health Work Plan 2021-2025 (National People’s Congress (NPC) representatives, Chinese People's Political Consultative Conference (CPPCC) representatives at provincial level, provincial governor‘s offices, municipal governor’s offices, provincial/district/municipal authorities of the Ministry of Civil Af-fairs, provincial/district/municipal authorities of the Ministry of Health, provincial/district/municipal authorities of the China Disabled People’s Federation).

Justification of the final evaluation

Based on HI’s internal Project Quality Policy (PQR), this assessment will serve as the final external evaluation and will be carried out through a participatory process, involving HI and its small grant activities implementing partner Changsha Heart Wing Club and Chongqing Dong Qing Centre to ensure their active engagement and ownership of the evaluation process.

The COVID-19 pandemic occurred concurrently within the project period and had underestimated impact on the project implementation. Identification of lessons learned and good practices is currently ongoing and can be used as resources for the consultant.

The evaluation conclusions and recommendations are primarily intended for the project team, which includes HI staff, partner staff, and partner board members. The findings will also be shared within HI to assist in decision-making, capitalizing on the experience and building upon it. Additionally, the evaluation report will be submitted to the project funding agency, BftW, as part of the contractual obligation and accountability.

2. Objective of the Consultancy

The objective of the consultancy is to carry out the final evaluation of the Promoting the dignity and fundamental freedom of people with mental health disabilities project (MH)  and provide an independent assessment of the level of achievement of the indicators, outputs, outcomes and impact of the intervention.

  • The evaluation at HI aims to ensure quality, learning and accountability. It should be conducted according to the criteria and approaches outlined in HI’s Project Quality Framework (HI will provide the E-copy of this document to the consultant).
  • The evaluation will be conducted in light of the COVID-19 context, considering any impacts that the pandemic may have had on the project’s implementation and outcomes, as well as any adaptations made by the project team in response to the situation.
  • Recommendations from the evaluation should be based on evidence, strongly linked to the evaluation findings and provide clear strategic-level guidance to HI, HEART WING AND DONG QING on how to address them and improve future programming.

More specifically, the consultant is expected to provide an analysis of the quality of the project based on the following criteria and evaluation questions:

Evaluation Criteria

Evaluation Questions

1.      Relevance

(Needs, context, lessons learned)

How well did the project meet the needs and align with the development priorities of the Government of China and local authorities as well as the priorities of businesses in the target areas, and take into account the available capacities and expertise at the national and provincial levels?

How did the project maintain consistency with other ongoing interventions in the field of Mental disorders social inclusion in the target areas? Has the project learned from experience throughout the project cycle? What lessons can be drawn from the project’s engagement between CSOs and persons with mental disabilities in promoting disability inclusion in and around the community, particularly in the context of COVID-19 pandemic, and how can these lessons inform future efforts in this area of intervention?

2.      Effectiveness

(Results, adjustment, technically)

To what extent have the project outputs been achieved, considering challenges and delays caused by the COVID-19 pandemic? Do the results obtained contribute to the achievement of the project objective?

How did the project demonstrate flexibility in adapting to changes in the local context and unforeseen circumstances, such as the COVID-19 pandemic?

Is the technical quality of the project achievements in line with HI's technical standards and/or international standards?

3.      Efficiency

(Skills, optimisations, responsiveness)

To what extent have the necessary skills and expertise (human, logistical, financial, technical) available enabled the project to achive its objectives?

How was the division of tasks and responsibilities between HI and HEART WING AND DONG QING defined and managed clearly and how effective was this collaboration in achieving the objectives of the project, particularly in light of the COVID-19 pandemic?

How effective were the CSOs in their role of engaging with community-based rehabilitation and promoting mental disorders social inclusion practices?

4.      Changes

(Effects, empowerment)

To what extent did the project enhance the capacities of various stakeholders on mental disorder inclusion, including CSOs, community members and government officials, despite the challenges posed by the COVID-19 pandemic? The result will produce significant positive changes in the lives of persons with mental disabilities sustainably.

Does the project contribute to the empowerment of actors, target populations and services?

3. Methodology

3.1 Collection method 

The method of data collection will be determined by the evaluator. However, a mixed-method approach is highly recommended to complete a comprehensive assessment of the project’s evaluation criteria (above).  It is recommended that the data collection methodology include the following:

  • Review of the available internal reports of the project
  • Individual interviews with senior staff from HI and small grant activities implementing partners
  • Individual interviews and/or focus group discussions with project target groups and key stakeholders (staff of local CSOs and family members)
  • Good case studies/success stories of beneficiaries: This will be conducted through a face-to-face interview (male and female).

The consultant is expected to provide a detailed methodology for the evaluation, which should include the following activities (but are not limited to):

Desk Phase and inception report

5 working days

·Review existing project documents and other relevant documents.

·Briefing with HI Focal Persons, including HI project team, operations coordinator, country director and regional/HQ technical specialists.

·Organize a bilateral meeting with HEART WING AND DONG QING the small grant activities implementing partner.

·Develop the inception report for HI’s feedback and approval. This will include adjusting the evaluation questions if needed, developing the evaluation matrix, planning for primary data collection with the list of stakeholders to meet during the field phase along with the data collection methods, tools and sampling methodology and updating the workplan.

·Coordinate with HI for the translation of the tools from English to Chinese.

Field Phase (in-person)

10 working days, including travel days

The consultant will detail the steps and methodology for HI’s feedback and approval. This may include surveys, semi-structured interviews and/or focus group discussions.

The consultant will take into consideration that the data collection process may be constrained by limited access to the project sites and the availability of stakeholders to participate in interviews or other data collection activities.

Reporting Phase

5 working days

·Conduct an analysis of the data collected in accordance with the evaluation criteria and questions.

·Organize a debriefing workshop with the project team to share the preliminary findings in a PowerPoint presentation and gather feedback and clarification as necessary.

·Submit the draft report within 10 calendar days after the field phase for feedback and comments from HI and HEART WING AND DONG QING.

·Submit the final report, incorporating the feedback and comments received from HI and HEART WING AND DONG QING. The report should not exceed 25 pages (excluding Annexes) and will be accompanied by raw data and any supporting documents.

HI will assist the consultant with logistics and inquiry during the field visit, which includes arranging a workspace, scheduling meetings upon request, providing Chinese translation and interpretation support by an external translator, and offering local transportation. HI will also ensure that all relevant documents are up to date and easily accessible (in electronic form) by the evaluator from the first day of the contract (desk phase).

3.2 Actors involved in the evaluation

  • A steering committee will comprised of members of implementing partner,s the Regional MEAL Manager, technical specialists, China Country Manager/Operations Coordinator and the project manager in China.

Key steps

Minimum Responsibilities

PHASE 1 & 2

Preparation & framing

Drafting of ToRs

The Steering Committee will guide and validate the ToRs drafted by the person in charge of the evaluation.

Scoping Meeting and Inception Report

The Steering Committee will validate the choice of evaluator and ensure the impartiality of the selection.

PHASE 3

Implementation

Presentation: evaluation findings and recommendations

The Steering Committee must participate in the scoping meeting (methodology, expected results...) and validate the inception report for future steps.

End-of-evaluation Questionnaire

The Steering Committee and the evaluator ensure that they have a common understanding of the conclusions & recommendations expressed.

The Steering Committee will provide elements that allow the evaluator to refine his/her recommendations *.

Quality of the final report

The Steering Committee must participate in filling in the end-of-evaluation questionnaire.

PHASE 4

Valorisation

Final report

The Steering Committee will specify the quality aspects expected from the evaluator for the final report, and provide feedback on the draft report.

Elaboration of an action plan and follow-up of the recommendations.

The Steering Committee will provide feedback on the draft report and then validates the quality** of the evaluator's final report.

4. Principles and Values

4.1 Safeguarding and Anti-Corruption Policy

The consultant will have to respect HI’s following code of conduct and safeguarding policies (HI will provide the E-version as attachment of the contract):

  • Code of conduct:
  • PSEAH policy:
  • Child protection policy:
  • Anti-fraud and anti-corruption policy
  • Disability Gender & Age (DGA) policy

4.2 Ethical measures*

As part of each evaluation, HI is committed to upholding certain ethical measures. It is imperative that these measures are taken into account in the technical offer:

  • Guarantee the safety of participants, partners and teams: the technical offer must specify the risk mitigation measures.
  • Ensuring a person/community-centered approach: the technical offer must propose methods adapted to the needs of the target population (e.g. tools adapted for illiterate audiences / sign language / disability-friendly materials, etc.).
  • Obtain the free and informed consent of the participants: the technical proposal must explain how the evaluator will obtain the free and informed consent and/or assent of the participants.
  • Ensure the security of personal and sensitive data throughout the activity: the technical offer must propose measures for the protection of personal data.

*These measures may be adapted during the completion of the inception report.                   

5. Deliverables & Timeframe

5.1 Deliverables

The expected deliverables of the evaluation assignment are:

  • An inception report that includes a detailed evaluation methodology, workplan, and data collection tools. The report should also outline the evaluation criteria and questions and provide a detailed explanation of the sampling strategy. The inception report and tools should be validated by HI before starting the field phase.
  • A PowerPoint presentation outlining the preliminary findings, conclusions and recommendations. The presentation will be used to facilitate the debriefing workshop with the project team.
  • A final report (max. 25 pages) that presents the evaluation findings, conclusions, and recommendations. The report should be structured according to the evaluation questions and criteria, using HI’s template. The report should be written in clear and concise language and provide a detailed analysis of the data collected during the evaluation.
  • Raw data sets from the evaluation, including any qualitative and quantitative data, as well as copies of all documents and reports generated during the evaluation process.
  • An end-of-evaluation questionnaire that must be completed by the consultant and HI to outline their thoughts on the quality of the evaluation and its usefulness for future programming.

The consultant will provide all the deliverables in electronic format in English. The final evaluation report will remain the intellectual property of HI.

5.2. End-of-Evaluation Questionnaire

An the end-of-evaluation, a questionnaire will be given to the evaluator. This questionnaire will be completed by him/her, a member of the Steering Committee and the person in charge of the evaluation.

5.3 Evaluation date and schedule

The evaluation consultancy is expected to last 20 working days, excluding days of in-between time required for HI’s feedback and validation of the inception report and evaluation report.

  • The consultant is expected to be contracted by end of July 2023.
  • The field phase in China is scheduled to take place by end of Aug 2023, and the evaluation visits will be planned in collaboration with the project team (HI and HEART WING AND DONG QING) and subject to the activities planned for the proposed timeframe.
  • The deadline for the submission of the draft evaluation report for feedback and comments from HI and HEART WING AND DONG QING is Sep 30th, 2023.
  • The final report, incorporating feedback and comments from HI and HEART WING AND DONG QING, should be submitted by Oct 10th, 2023.
  • A final debriefing session will be held with HI, during which the end-of-evaluation questionnaire will also be completed.

6. Resouces

6.1 Expertise requirement from the consultant(s)

The consultant must meet the following requirements:

  • At least 5 years of relevant professional experience in the field of mental disability inclusion, community-based rehabilitation or related fields.
  • Proven experience in conducting external project evaluations, including in qualitative and quantitate data collection and analysis.
  • Experience working with CSOs, DPOs, the private sector and other stakeholders in the mental disability sector.
  • Familiarity with disability rights and inclusion, including the UNCRPD and SDG, as well as the business case for engaging people with mental disabilities.
  • Demonstrated knowledge of the Chinese context and cultural sensitivity.
  • Excellent report writing skills in English.
  • Availability to travel to China for the field phase.

6.2 Budget

The consultant should submit HI the financial proposal including very costs to produce the identified deliverable services (transportation, accommodation, personnel, translation, computer, printing questionnaires, specific tools/software, office materials…) and any applicable tax in the country (VAT or withholding tax).

The proposed payment modality is as follows:

  • 25% upon signature of the service contract
  • 25% after the validation of the inception report
  • 50% after the receiving of all deliverables and validation of the final evaluation report.

If the consultant requests a different payment method, they must provide a justification for it in the proposal. Please note that no per diem will be paid to the consultant, and they will be responsible for their own security in all countries, HI will not cover any insurance fee during the consultancy period.

Please note that the final payment is conditional on the validation of the final evaluation report and not solely on its submission. Validation means ensuring that the report meets the quality standards as per the checklist attached above, and it is not based on the appreciation of the project evaluated.

6.3 Resources available to the evaluation team (data, document, etc.)

Before the consultancy, HI will provide the consultant with all requested project documents and data (as available).  

7. Application Procedure

Proposals from interested consultant(s) must include the following documents (compulsory documents in English):

  1. An Expression of Interest/cover letter, including how the skills and competencies described in the Terms of Reference are met.
  2. A Curriculum Vitae detailing the consultant’s experience and qualifications to undertake the assignment.
  3. A technical proposal outlining the proposed methodology and a tentative workplan.
  4. A financial proposal in EURO that includes all costs to produce the defined deliverable work (e.g. transportation – international/local, accommodation, personnel, meals, travel insurance, translator, other administration costs during the field visit, etc.).
  5. An example of an evaluation report previously produced by the consultant.

Proposals must be submitted in English no later than (June 30th 2023) and should be sent electronically to Vannak TOUCH v.touch@hi.org and Leo LI l.li@hi.org.

Evaluation of proposals will be made by a Selection Committee in two phases:

  • Administrative selection: The committee will first check for completeness of the application and verify that all compulsory items listed above are included. Incomplete applications will not be considered for technical selection.
  • Technical selection: The best application will be selected based on the quality of the technical proposal, the competitiveness of the financial proposal, the skills and previous experiences of the consultant, and the demonstrated expertise of the applicant to successfully undertake the assignment.

Only candidates who pass the administrative selection will be considered for the technical assessment. Shortlisted applicants may be invited for an online interview.

HI reserves the right to contact the applicants for additional information or clarification before the final selection of the selection committee.

发布评论
文明上网理性发言,请遵守评论服务协议
表情
全部评论
最新 最热
加载更多