International Consultant-Final Project Evaluation-TVET 2-BAG

OVERVIEW

Parent Sector : Education Sector (ED)

Duty Station: Baghdad

Job Family: Education

Type of contract : Non Staff

Duration of contract : From 1 to 6 months

Recruitment open to : Internal and external candidates

Application Deadline (Midnight Paris Time) : 29.11.2023

 

UNESCO Core Values: Commitment to the Organization, Integrity, Respect for Diversity, Professionalism

 

Background

 

Supported by a $5.852 million grant from the European Commission, the Iraqi Ministries of Education and Higher Education and Scientific Research with UNESCO’s support have been implementing the “Jobs Creation through Technical & Vocational Education System Strengthening & Labour Market Linkages Programme (TVET Reform Phase II)” from January 2021 to December 2023. TVET II aims to strengthen vocation and technical education by embedding the new reform arrangements and structures while aligning the policies and procedures of the relevant ministries and local directorates to ensure coherent implementation.

This programme builds on progress made in previous years through TVET I implemented from 2015 to 2020. TVET I focused primarily on upstream policy-orientated work that laid the foundations for how TVET will be delivered in Iraq’s future. Under TVET II the Government of Iraq promoted the idea of Centres of Vocational Excellence (CoVES) and expressed interest in using the instruments of the Bologna process to increase the comparability of their qualifications with the qualifications of International Education systems. The programme is well-aligned with Iraq's national plan for education to support the programme towards the realisation of SDG 4 and SDG 8.

Through these reforms, Iraq intends to achieve inclusive access to skills development services and, contributing to improved employability and socio-economic status.

•    Component 1: To strengthen the Ministry of Education national and Directorate systems and capacities for quality management, curriculum and labour market alignment for vocational education.

•    Component 2: To enhance the performance of selected vocational schools to create and operate Centres of Excellence.
•    Component 3: To develop higher education institutions’ systems and leadership for meeting local skills needs and international recognition of qualifications.

 

Purpose and Rationale of the Final Evaluation

 

As established in the TVET II programme document, UNESCO will conduct an external evaluation of the programme within six months after the programme completion date of 1st March 2024.

The evaluation will focus on the entire implementation period from 1 January 2021 to 31 December 2023. The purpose of the evaluation will be primarily summative. The summative elements are for accountability purposes i.e. to define the projects’ performance (approaches, methods) and achievements.

The findings of the evaluation will be used by UNESCO for promoting the results of the programme with key Education stakeholders in Iraq, informing the donor and for future similar project design (TVET 3) and evaluation mainly for programme staff and partners. In addition, the key findings and recommendations will contribute to the knowledge generation and management of the Government of Iraq, the European Union (donor), and UNESCO.

The key intended users of the evaluation are:

•    Projects’ official stakeholders in Federal Iraq and KRG (PMAC, MoLSA, MoHESR, MoE, DoE, DG of Vocational Education, MoP; Project Steering Committee,  relevant KRG ministries, Presidents of the Technical Universities and Headmasters of the Vocational schools). The primary users of the evaluation will be UNESCO, The EU Delegation to Iraq and the Project Steering Committee.  The secondary users will be the MoP, the MoHESR/MoE, and the senior project staff within UNESCO.
•    Beneficiaries (vocational centres, vocational schools and technical/polytechnic universities; private sector, teachers and TVET leaders and learners).
•    UNESCO.
•    European Union/European Commission/EUD to Iraq (Donor).

Objectives

 

The overall objective of the final evaluation is to assess the achievements, outcomes and impact of TVET measured against its original aim and objectives. Furthermore, the evaluation aims to identify and document best practices and lessons learned, as well as generate recommendations for future programmes. More specifically, the evaluation aims to:

a.    The overall objective of the evaluation is to assess the level of project performance considering the project environment and thus implementation modalities. This will include:
i.    Using the following evaluation criteria: assess issues of relevance, coherence, effectiveness, efficiency and coordination; sustainability; visibility and outreach; 
ii.    Generate lessons learned and good practices resulting from the project with particular focus on the proposed Closing activities period with a focus on identifying key actions to enhance the sustainability of the project and the implementation future planned activities. 
iii.    To provide recommendations for TVET 3 concept note. 

b.    Assess the effectiveness in achieving the defined objectives and to what extent conditions are put in place that facilitate sustainability or a longer-term impact, i.e.  to feed into decision-making for future similar project design and implementation (mainly for programme staff and partners). The evaluation's findings/recommendations are expected to provide UNESCO with further guidance on the proposed 6 month “Closing Activities Period“ with a focus on identifying key actions to enhance the sustainability of the project. Implementation of future planned activities.

c.    Review the validity of the programme’s theory of change and implementation strategies and identify opportunities, challenges, good practices, and lessons learnt that will be useful for strengthening and enhancing similar interventions for the design of the potential next phase of the programme.

The evaluation shall also provide context in what are the potential future longer-term effects of the project on the TVET reform in Iraq and the relevant stakeholders and beneficiaries, as well as address the issues and assessment carefully detailed in this section.
 

Deliverables 

 

The expected deliverables under this contract (to be written in English) are:
a.    An Inception Report including the related evaluation matrix the refined evaluation questions.
b.    Draft evaluation report on the TVET II programme
c.    Final evaluation report on the TVET II programme

These deliverables are the responsibility of the external evaluator with inputs from UNESCO. Furthermore, the process will require a Kick-off meeting, ToC Workshop, a Validation workshop with the reference group and involvement of other relevant other relevant stakeholders as appropriate.  
 

Long Description Scope

 

The evaluation of the programme will be based on the OECD/DAC Evaluation criteria , namely relevance, coherence, efficiency, effectiveness/(signs of) impact and sustainability 
of the programme outcomes. The Contract will assess the implementation and management of the following interventions over the programme’s entire implementation period:

The TVET II programme will be assessed against the below evaluation criteria. The questions are indicative and shall be refined during the inception phase of the evaluation.

Relevance  

i.    To what extent were the objectives of the project relevant in addressing the challenges of the TVET sector?

 

Coherence

ii.    To what extent were the programme's interventions consistent with the priorities of the technical and vocational national policies of Iraq?

 

Effectiveness

iii.    Did the project design, and chosen approaches, as well as methodologies applied by TVET 2 project address the development needs of the Iraqi TVET sector administration effectively?
iv.    How does the beneficiary safeguard the sustainability of project outcomes and outputs towards an improved capability in applied and strategic TVET management?
v.    Are the principles of gender equality and inclusion effectively mainstreamed in the implementation of all aspects of the programme as per the programme design? Has the programme achieved results as set out in the programme document that are gender responsive? 
vi.    To what extent did the project consider the needs of disadvantaged or marginalized groups, including people with disabilities? 

 

Efficiency

vii.    Considering the given/prevailing project framework conditions on the ground in Iraq, did the chosen mode of delivery under the TVET 2 project provide/allow for an efficient implementation process?  

 

Signs of /Pathways towards Impact

viii.    Did the project design, and chosen approaches, as well as methodologies applied by TVET 2 project address the development needs of the Iraqi TVET sector administration effectively?
 

 

Sustainability

ix.    To what extent are the benefits of the programme likely to be sustained or taken forward by the government or development partners after the completion of the programme? 
x.    What measures have been taken to ensure ownership and commitments of key stakeholders and partners in support of financial, political and institutional sustainability? 
xi.    What changes have the programme established to ensure that the results and impact of the programme are sustained beyond the programme timeframe?

 

Visibility, Communication and Knowledge management 

xii.    To what extent was the programme’s visibility and communication adequate?
xiii.    To what extent have the programme activities and results been communicated in an effective manner to provide the required visibility and recognition of the programme, to attract future funding opportunities? 

Design and methodology

Under the overall supervision of UNESCO Representative to Iraq, Head of Education and the direct supervision of the TVET II Programme Manager, the Contractor will assess the programme’s interventions according to the above evaluation criteria: Relevance, coherence, effectiveness/signs of impact, efficiency, and sustainability. Cross cutting issues such as gender equality and accountability will also be part of the analysis.

In line with UNESCO’s Evaluation Policy (2022-29), the evaluation will have to comply with the United Nations Evaluation Group (UNEG) Norms and Standards for Evaluation, UNEG Guidelines for Integrating Human Rights and Gender Equality in Evaluations and UNEG Ethical Guidelines for Evaluation. The evaluation team will also have to ensure that ethical, human rights and gender equality principles are duly integrated at all stages of the evaluation process. 

The Contractor shall propose an approach and detailed methodology ensuring an evaluation framework that allows to rigorously investigate the above evaluation questions. The final methodology, as well as the evaluation framework including the list of evaluation questions, shall be adopted after the inception phase and after the approval of the inception report by UNESCO.

The Contractor is expected to perform, but not limited to, the following tasks in line with the approved methodology:
 

a.    Desk review of project-related documentation and other relevant documents to facilitate the formulation of the evaluation questions. Review of the log-frame/results framework, annual and progress reports, mid-term ROM evaluation and other related materials; assessment of results and deviations from planned activities and outputs.
b.    Stakeholder analysis and sampling framework.
c.    Validation of the overall Theory of Change for the project including the results pathways and its underlying assumptions.   

d.    Key informant interviews and/or focus group discussions with relevant stakeholders and UNESCO staff, including key partners and beneficiaries in Baghdad, Basra, Karbala, Ninawa, and Kurdistan Region of Iraq (Erbil, Sulaymaniyah, Dohuk), members of the project steering committee, training participants, the European Union delegation in Baghdad. 
e.    Questionnaires and surveys addressed as appropriate to groups of stakeholders at various levels.
f.    Selected field visits to meet with the local stakeholders, beneficiaries and partners involved in the implementation of the projects, including interviews with direct beneficiaries. 
g.    Participatory workshops to steer the evaluation and to discuss findings and recommendations.
h.    The Evaluation will involve a visit to Iraq and travel to the locations of the project activities.  While a considerable amount of the consultations can be conducted remotely (for example via ZOOM)   one in-country visit is considered as essential.

i.    Submit the following deliverables to UNESCO for approval:

 

Deliverable 1: Inception Report, (10 pages maximum excluding annexes) prior to visit to Iraq will outline in detail the key scope of the work; a work plan/schedule of tasks; evaluation methodology, refined design and questions; proposed methods; sources of data; data collection plan (proposed list of people to be interviewed), as well as an evaluation matrix showing the relationship between the evaluation questions and methods of data collection and analysis.

Deliverable 2: First draft of Evaluation Report (30 pages excluding annexes), which will be structured as follows:

Derivable 2 includes a Kick-off meeting for discussion and presentation of the assignment and objectives, a ToR workshop to discuss and refine the ToR, and a validation workshop to present and discuss the findings and preliminary recommendations.  The meetings can be conducted in person or virtually.

The Report content should at least follow the following:
 

•    Executive Summary (maximum 10 pages) the encapsulates the essences of the key-finding, conclusions, lessons learnt.
•    Introduction, including programme description and Theory of Change 
•    Evaluation purpose

•    Evaluation methodology
•    Findings
•    Lessons learnt
•    Conclusion
•    Recommendations
•    Annexes (including the list of stakeholders consulted during the evaluation, key documents reviewed, ToR, detailed methodology and sampling approach, interview/ focus group discussion protocols, survey forms and aggregate results, budgetary analysis, biodata of the evaluators.)

The Contractor shall also prepare a slideshow presentation of the findings and recommendations (in Microsoft Powerpoint format), with accompanying summary notes enclosed within the slideshow presentation or as a separate document (in Microsoft Word format) in English and translated into Khmer. 

Deliverable 3: Final Evaluation Report of no more than 45 pages (excluding annexes) incorporating the appropriate comments and suggestions provided by UNESCO and MoEYS on the draft final report. In addition, the Contractor shall prepare a summary report translated into Arabic highlighting key findings and recommendations, which will be no more than 10 pages.

The methodology must consider the participants’ safety throughout the evaluation process as well as research ethics and quality assurance. The evaluation will follow UNESCO’s evaluation policy and will be conducted in accordance with the principles outlined in the UNEG ethical guidelines for evaluation.

Timing and payment

 

The fee is payable in the following proposed payment installments upon certification by UNESCO of satisfactory performance by the Contractor of the work corresponding to each payment. All payments shall be made by bank transfer. UNESCO shall be responsible for its own banking fees but any possible intermediary banking fees, as well as the beneficiary's own banking fees, shall be the responsibility of the contractor. As such, please take these banking fees into account when preparing the financial proposal/price schedule.

Remuneration will be based on the submission and validation of the deliverables. All submitted reports/materials should be drafted in English.

The following timeframe is tentative, and to be finalized during the inception report. The duration is expected to be 9 weeks. 
 

 

No. Deliverable Tentative due date Amount payable 1 Kick off meeting, desk research, and Inception report 1st – 2nd week 20% of the contract value 2 First draft of Evaluation report

3rd – 6st week

40% of the contract value 3 Final Evaluation report 8th – 9th week 40% of the contract value

 

Roles and responsibilities for the evaluation

 

UNESCO Iraq Office is responsible for the overall management of the evaluation and technical oversight of the deliverables, under the direct supervision of the TVET-II Programme Manager.

1)    Responsibilities of the evaluator include but are not limited to:
a)    Treating documents in a confidential manner;
b)    Returning all documents to UNESCO;
c)    Asking for permission before giving any information on the evaluation to third parties;
d)    Providing all logistics such as office space, telecommunication, the printing of documentation, etc.
e)    Ensuring availability for a briefing via teleconference with the TVET-II team before in-country data collection
f)    Ensuring availability for team members to partake in in-country missions of sufficient duration for all needed data collection
g)    Situating the final evaluation of the programme within a holistic understanding of the context in which it operates;
h)    Maintaining positive relationships during all stakeholder interactions;
i)    Ensuring that the final evaluation and all interactions are grounded on ethical principles and cultural sensitivity;

2)    Responsibilities of UNESCO include but are not limited to:
a)    Providing key documents for desk review, as well as during data collection as needed;
b)    Providing contact information and facilitating access to stakeholders and the participation of stakeholders during data collection;
c)    Ensuring availability of staff to cooperate with the evaluation team;
d)    Arranging field visits and supporting organisation of logistical arrangements during field visits for the evaluation team;
e)    Liaising with the Evaluation Reference Group to provide updates on evaluation progress;
f)    Liaising with the Evaluation Team and Evaluation Reference Group to schedule meetings and set deadlines for feedback;
 

3)    Evaluation reference group 
If determined convenient by TVET II’s programme manager during the inception phase of the evaluation, an evaluation reference group shall accompany the evaluation process and provide feedback on the Inception Report, data collection tools, preliminary findings, and draft evaluation report. 

The reference group will include representatives from UNESCO other relevant group determined by the Programme Manager of TVET-II during the inception phase. The reference group shall exchange/meet periodically and be consulted in the different stages of the evaluation, as appropriate. 
 

COMPETENCIES (Core / Managerial) Accountability (C) Communication (C) Innovation (C) Knowledge sharing and continuous improvement (C) Planning and organizing (C) Results focus (C) Teamwork (C) Professionalism (C) - For detailed information, please consult the UNESCO Competency Framework. Required Qualifications:

 

•    Advanced university degree in the field of education, political and social sciences, or any other relevant field.
•    Demonstrated expertise in the field of education at national and international levels (acquired through education, and training ( a minimum of 5 relevant work experiences).
•    At least 10 years of relevant experience in designing, conducting, and leading evaluations of Education Projects in MENA, including similar complex, large-scale international development projects in conflict settings. (at least three assignments).
•    At least 10 years of relevant experience in the education/training sector.
•    Excellent planning and reporting skills.
•    Good facilitation skills; 
•    Excellent oral and written communication capabilities; 
•    Excellent analytical and demonstrated drafting skills in English: the ability to collect and analyze information, synthesize ideas and feedback, and prepare reports in a clear and concise manner (demonstrated by at least three work examples). 
•    Ability to adapt to dynamic and challenging environments. 
•    Excellent knowledge of English (both written and spoken).

Desirable Qualifications:

 

•   At least three work experiences demonstrating understanding and Knowledge of the TVET sector.
•  Previous experience in Iraq.

•  At least three examples of work demonstrating understanding and application of UN mandates in Human Rights and Gender Equality and/or of gender- and culturally-sensitive approaches in evaluation.

•  At least two examples of work experience in the UN or experience with assignments for the UN in the framework of the Sustainable Development Goals - 2030 Agenda
•  At least three examples of work demonstrating familiarity with the Theory of Change approach and/or experience of developing ToCs in fields of social development 

•  knowledge of Arabic Language

Application Process

 

A.    Technical Proposal which should consist of:

1.    A description of your proposed approach and methodology for undertaking the assignment (5 pages only).
2.    Work plan, including description of key deliverables/ milestones (in Gantt chart)
3.    An up-to-date curriculum vitae

B.    Financial Proposal which should consist of:

showing the amount to be charged for the assignment expressed as an overall lump sum, which should be quoted in US Dollars only. Any travel and subsistence requirements should be indicated separately. Please design the payment scheme against a clear and specific set of deliverables and milestones as reflected in the work plan.
 

C. Other documents 
List of relevant Previous work assignments and three (3) contacts as references.

 

Selection and recruitment process

 

Please note that all candidates must complete an on-line application and provide complete and accurate information. To apply, please visit the UNESCO careers website. No modifications can be made to the application submitted.

The evaluation of candidates is based on the criteria in the vacancy notice, and may include tests and/or assessments, as well as a competency-based interview. 

UNESCO uses communication technologies such as video or teleconference, e-mail correspondence, etc. for the assessment and evaluation of candidates.

Please note that only selected candidates will be further contacted and candidates in the final selection step will be subject to reference checks based on the information provided.

Footer

UNESCO recalls that paramount consideration in the appointment of staff members shall be the necessity of securing the highest standards of efficiency, technical competence and integrity. UNESCO applies a zero-tolerance policy against all forms of harassment. UNESCO is committed to achieving and sustaining equitable and diverse geographical distribution, as well as gender parity among its staff members in all categories and at all grades. Furthermore, UNESCO is committed to achieving workforce diversity in terms of gender, nationality and culture. Candidates from non- and under-represented Member States (last update here) are particularly welcome and strongly encouraged to apply. Individuals from minority groups and indigenous groups and persons with disabilities are equally encouraged to apply. All applications will be treated with the highest level of confidentiality. Worldwide mobility is required for staff members appointed to international posts.

UNESCO does not charge a fee at any stage of the recruitment process.