Terms of Reference for a Consultancy to Conduct End of Project Evaluation
Chipata, Katete, Petauke, Choma, Pemba, Livingstone, Kazungula, Zambia, Full Time Salary:
Project Name: Open Contracting for the Health (OC4H)
1. Project Background
Each year governments around the world spend vast sums of public funds on health related contracts for everything from medicines to hospitals. Public procurement within healthcare is extremely complex and often opaque. This makes it highly vulnerable to undue influence and misuse, with severe implications for global health and economic development. Effective procurement policies can help ensure that high quality, cost effective products are purchased at the right time, in the correct volume, at the right price and deliver to the health needs of the local population.
Due to the scale of public expenditure and its direct impact on local communities, health is an ideal sector to demonstrate the benefits of open contracting practices in public procurement. The growing consensus among governments, civil society and the private sector on the potential of open contracting has been demonstrated through the commitments made at the 2016 Anti-Corruption Summit in London where countries including Argentina, Malta, Mexico and Nigeria all made specific reference to the health sector in their national action plans. Open contracting also aligns with the commitment made by the African Union, in January 2018, when it recognized corruption as hampering “efforts aimed at promoting democratic governance, socio-economic transformation, peace and security and the enjoyment of human rights in the AU member states”, and declaring 2018 “African anti-corruption year”.
Overall, open contracting has been shown to significantly contribute to the strengthening of health systems, complementing the efforts of governments, international donors and institutions to build efficiencies and better meet the needs of the population. Open contracting can minimize public procurement vulnerabilities, and has the potential to allow health systems to develop the resilience needed to withstand health crises, such as the Ebola outbreak in West Africa, and also the recent outbreak of COVID 19. In different contexts (e.g. Colombia and Ukraine), open contracting has also proved to benefit small and medium enterprises (SMEs), by contributing to fairer and more competitive markets that have in turn benefitted the service users.
Based on the above background, in 2018, the Open Contracting for Health (OC4H) project was developed by Transparency International Global Health programme (TIGH). The project is funded by the UK Government’s Foreign, Commonwealth & Development Office (FCDO). The OC4H initiative aimed at improving healthcare systems through the principle of open contracting in healthcare procurement, building on existing efforts by government, civil society and the private sector (the ‘golden triangle’) to identify opportunities for open contracting in the health sector. OC4H worked with key partners to make procurement more transparent and accountable by making a step towards open contracting as the default process for public healthcare procurement, in order to achieve better functioning health systems and outcomes in Zambia. The project worked with public procurement officials to strengthen open contacting data at a local level. The project also worked to increase disclosure of contracting information, improve public participation in contracting processes and collaborative engagement between government, civil society and private sector. The project strengthened the capacity of community monitors and civil society to collectively and effectively demand accountability and value for money in public contracting health sector.
2. Project Description
In partnership with Transparency International Global Health programme, TI-Z has been part of a global initiative that seeks to reduce corruption in the healthcare sector, and improve global health and healthcare outcomes through a project is known as the ‘Open Contracting for Health (OC4H). OC4H is a three-year project which will come to an end on 31/03/2021. The project aimed to improve healthcare systems in a number of countries across Sub-Saharan Africa and Asia, through the principle of open contracting in healthcare procurement. The project aimed to enhance performance of contracts through increasing contract information disclosure and citizen monitoring with a view of addressing challenges that negatively impact on the quality, schedule and cost of contracts in Zambia. Open contracting is the practice of publishing and using accessible procurement cycle information to ensure that the vast sums of public money are spent honestly, fairly and effectively. This approach uses government transparency to foster participation between public bodies, businesses and civil societies to boost the integrity, fairness and efficiency of public contracting. The project is being implemented in seven districts in Eastern and Southern Provinces of Zambia i.e. Chipata, Katete, Petauke, Choma, Pemba, Livingstone and Kazungula.
3. Project Development Outputs, Outcomes and Impact
The three main outputs for the OC4H project are:
National health systems have the skills and resources needed to implement open contracting in public procurement
Supplier diversity in health sector public procurement is facilitated
Civil society is sustainably engaged in public procurement processes
The outcome of the OC4H project is:
Public procurement in national health systems is made more transparent The expected impact of the OC4H project is:
Health outcomes in partner countries have improved
The full Log frame can be found in Annex A.
FCDO – Funder
TIGH – Project Lead
• Transparency International Zambia (TI-Z) – Implementing organization
The Procurement Officers and District Health Officers in all 7 districts – Local Governments
Ministry of Health (MoH)
Zambia Public Procurement Authority (ZPPA)
Private Sector across all districts (SMEs)
5. Methodologies used by the project
Contract and public infrastructures monitoring (construction and up grading of Health Centres)
Capacity building for procurement officers, private sector actors (SMEs) and CSOs
Review / Feedback meetings with all stakeholders
Use of online tool (e-GP)
6. Key activities conducted
• Build alliances between various national stakeholders
An OC hub designed to build capacity around the production, analysis and use of data
Training for government procurement officials to introduce open contracting standards
Facilitation of peer-to-peer learning
Support to civil society to independently and sustainably monitor the use of open contracting standards
Collection of evidence and data throughout the implementation of the project and the dissemination of learning
Hosting national/ district workshops
7. Purpose and Scope of the Evaluation
The main goal of this consultancy is to carry out the final evaluation of the TI-Z project funded by FCDO under the umbrella of Transparency International focusing on the assessment of its relevance, effectiveness, efficiency, impact and sustainability, while paying attention to context and processes and learnings
8. Specific objectives of the evaluation
Generate learning and knowledge about the conditions in which the project achieved and may sustain its results in the context of open contracting principles
Show the results and social return on investment made in the project. This should be done in a credible and transparent way
9. Scope of Evaluation
The scope includes content, geographical and time scopes.
a) Content scope
The content scope for Evaluation is determined using OECD-DAC criteria for evaluation. Relevant criteria are associated with a number of key questions that are to be addressed and explored. The evaluation will also cover the analysis of the process of implementation, the changes that have occurred because of the project’s intervention, opportunities and constraints that have been encountered, important lessons that have been learnt and recommendations for future design and implementation for TIGH and TI-Z.
b) Geographical scope
The geographical scope will include; the 7 districts namely Chipata, Katete, Petauke, Choma, Pemba, Livingstone and Kazungula are the districts targeted by the project, as well as the Central government entities such as Zambia Public Procurement Authority (ZPPA) and Ministry of Health (MoH), CSOs and SMEs.
c) Time scope
The evaluation shall be expected to complete within 20 days, between 1st March and 25th March 2021.
10. Evaluation Criteria and Key Questions to Be Addressed in the Evaluation:
The following provides a guide to the questions to be addressed by this evaluation, under each of the criteria below:
Long term outcome/Impact
a. Did the OC4H project contribute to the intended impact of improved health outcomes within the country?
b. Did the OC4H project achieve the intended outcome within the country?
c. What positive or negative unintended outcomes resulted from the project?
a. Have the OC4H projects activities been effective? Have some been more effective than others, e.g. training vs advocacy?
b. Of the three outputs/target stakeholders which was most effective in contributing to the outcomes? Are there any key lessons relating to these?
c. How can Procurement Data be better linked to social accountability activities and other decision making transparency processes such as contract monitoring?
d. How can Procurement Data be better used by formal accountability/oversight institutions such as ZPPA, audit offices and anti-corruption departments?
e. How effective was the program M&E System in contributing towards effective management and quality implementation of the program activities? Is there evidence to show that information emanating from the monitoring system was adequately documented, reviewed, shared and utilized to improve management decision making and quality of program implementation at all levels of the project?
f. Is there evidence to show that information emanating from the benefiting CSOs and privates sector actors was adequately documented, shared and utilized to improve the quality of program implementation?
g. What key challenges (internal and external) to implementation were encountered and how effectively were these responded to? What can the project learn from these challenges that can help future interventions of TI-Z?
a. To what extent was the project aligned to the immediate needs and priorities of the target beneficiaries (including government agencies, other CSOs, private sector, citizens and Local government districts) as well as the priorities of national and local government agencies?
b. To what extent were agencies such as ZPPA, Ministry of health, and other CSO etc. involved in the project design and implementation process?
c. To what extent did the project adequately respond to needs/issues raised by the project beneficiaries (issues concerning public procurement, citizen participation, citizen’s feedback and social accountability issues)?
d. How appropriate were the alternative solutions/changes that was proposed by TI-Z help to improve the situation in public procurement processes?
e. What were the influencing strategies put in place by the project to address the issues concerning open contracting that affect service delivery to the citizens?
a. Where will the lasting impacts be, what are the major opportunities that were missed, and what other opportunities are there?
b. Can the program make a reasonable case in terms of value for money considerations informed decisions on financial expenditures on project inputs and activities with a view to maximizing program outputs?
c. Were there adequate resources to achieve the desired outputs and outcomes?
d. How did the project use resources for implementation? Could the use of resources be improved?
a. Does the project have an exit plan?
b. During the implementation of the project what have the target beneficiaries done as a result of the project and will continue to use even when the project is no more?
c. To what extent has the program developed local capacities, linkages and plans for ensuring that the effects of the different interventions can be sustained?
d. How has OC4H embedded itself within national CSO and development networks? Was there any benefit to this, and what potential is there to continue to utilize in-country networks beyond the lifespan of the project?
e. How has the project been embedded into the health sector in the country? Is it possible to demonstrate that the project has contributed to impacting the performance of health services delivery?
Optional Additional Questions
These questions are not essential to the evaluation, however if an evaluator feels they can address all the above questions as well as examine the following it would be a welcome addition.
a. How can health information systems be used to inform procurement plans?
b. How could verification of stock of health commodities/medicines at the service level be used in conjunction with procurement data and/or contract monitoring in order to further accountability in the health sector?
It is expected that the evaluation will be carried out in conformity with evaluation best practices. The methodology will be defined by the consultant as it suits the scope of the evaluation.
The estimated duration of the assignment is 30 days. The first draft of the report must be submitted by 25th March, (to go through an internal review process) and the final report with comments addressed must be submitted on or before 9th April.
13. Reporting requirements/deliverables
In the course of the assignment the evaluator shall provide the following outputs in English:
a. Inception report outlining a detailed plan, methodology and timeline of activities to be accomplished by the evaluator under the assignment. This should be submitted in the form of a presentation to the TI-Z project team.
b. Interim report, to inform the project team of preliminary results.
c. Final evaluation report that answers the key evaluation questions, outlining persons/institutions interviewed, data collected, lessons learned and recommendations for future projects, consulted
and validated with TI-Z project team. This report will be submitted on 9th April 2021 taking into consideration the consultant addressing comments made by the TI-Z/TIHI team.
The consultant will be provided support by both TIGH and TI-Z to provide additional context about the project as well as identifying and putting into contact with relevant external stakeholders for potential interviews.
15. Application procedure
All expressions of interest should include:
a. Letter of interest (maximum one page)
b. Technical proposal highlighting: brief explanation about the consultant’s profile, consultant’s understanding of the TOR and previous experience in similar assignments.
c. Financial Proposal: the financial proposal should provide cost estimates for services to be rendered including daily profession fees and incidence expenses
16. Qualification and competencies
To accomplish the objectives of the end line evaluation, the resource person/consultant should have the following key qualifications and competencies.
a. The evaluator should be a reputable consulting company or an individual that shall be selected on basis of the knowledge and experience in the monitoring and evaluation field.
b. At least 5 years of professional experience in using evaluation methods of similar projects.
c. At least 5 years’ experience in conducting similar studies, financed by international financial institutions such as World Bank/USAID, DFID among others.
d. Lead consultant should have at least relevant Master’s degree with bias in Monitoring and Evaluation.
e. Knowledge of theory of change will be considered an asset.
f. Knowledge on Open Contracting is an added advantage
g. Knowledge of the governance, transparency and accountability programme.
h. Excellent oral and written English
17. How to Apply:
All suitably qualified and interested consultants should submit expression of interest that includes technical and financial proposal as well as, Curriculum Vita and contact information for three professional referees not later than 18th February 2021 at 5:00pm to email@example.com
The complete application packet must be submitted in the required format with the required attachments.
a. Technical Proposal
b. Technical Budget and budget notes
Offers received after the specified date will be considered late and will be considered only at the discretion of TI-Z. TI-Z reserves the right to make an award based on initial submission. TI-Z reserves the right to make no award if it is determined that the offers submitted do not satisfy the needs of the organization. All written proposals submitted must be valid for a period of not less than sixty (60) calendar days from the stated closing date.
18. Notification of Selection
TI-Z will notify the offered who submitted the highest scoring proposal in writing by email. Clarifications and revision of minor errors and omissions may be requested. Upon completion of either, offered may be required to submit a revised quote.
All prospective candidates are advised to follow the application procedure and guidelines provided.
Please note: Due to the current coronavirus pandemic we will not be accepting hard copy application submissions. Thank you.
TI-Z will not reply to any phone enquiry and that will lead to automatic disqualification
Annex A – OC4H Log frame
IMPACT Impact Indicator 1.1
Health outcomes in 2 partner countries have World Health Organization health surveillance
improved statistics have improved
Impact Indicator 2.1
Costs avoided through open contracting are identified,
compared to a procurement cost benchmark and
OUTCOME Outcome Indicator 1
Public procurement in national health systems is made Open contracting principles are applied in partner
more transparent country public health systems
Outcome Indicator 2
Data generated by Open Contracting can effectively be
used to report instances or trends of inefficacies and
vulnerabilities that may reveal trends or instances of
OUTPUT 1 – GOVERNMENT Output Indicator 1.1
National health systems have the skills and resources The Open Contracting for the Health Sector hub is
needed to implement open contracting in public made available and adapted to the respective context
Output Indicator 1.2
Relevant national procurement staff’s capacity is
improved in relation to open contracting principles
Output Indicator 1.3
Government actively publish tender documents to an
open and transparent platform
OUTPUT 2 – PRIVATE SECTOR Output Indicator 2.1
Supplier diversity in health sector public procurement is Private sector entities, including SMEs, are regularly
facilitated engaged with meetings and
activities around open contracting and transparency
in public sector procurement
Output Indicator 2.2
Potential contractors, including SME’s, utilise open
contracting information for public procurement
Output Indicator 2.3
Potential contractors have increased capacity to access
and utilise open contracting public procurement
OUTPUT 3 – CIVIL SOCIETY Output Indicator 3.1
Civil society is sustainably engaged in public Civil society monitor public procurements in health,
procurement processes using both Open Contracting data as well as physical
Output Indicator 3.2
The Open Contracting for the Health Sector hub is
made available to civil society organisations and
adapted to the respective context
Output Indicator 3.3
Civil society’s capacity to advocate for, and use Open
Contracting information is increased.