CONSULTANTS TO CONDUCT ENDLINE EVALUATION OF THE STRENGTHENING EDUCATION AND TRAINING IN SOMALIA (SETS II) PROJECT

REQUEST FOR CONSULTANTS TO CONDUCT ENDLINE EVALUATION OF THE STRENGTHENING EDUCATION AND TRAINING IN SOMALIA (SETS II) PROJECT

1. SETS II Project Background

In its accordance with European Union and other international principles and conventions, the ongoing SETS II project was designed to recognize and endorse the central role of education in economic and social development in Somalia. Reference was made to the new European Consensus Development Agreement which provided the framework for a common approach to development policy to be applied by the EU institutions and Member States while fully respecting each other’s distinct roles and competences. According to the Agreement, responding to the educational needs of children, women and youth is crucial to promoting responsible citizenship, developing sustainable and prosperous societies, and boosting empowerment of otherwise excluded groups and youth employment, key parameters which have been used to design this intervention. Emphasis was also made on eradicating poverty, tackling discrimination and inequalities, and leaving no-one behind through provision of universal access to quality education and training which will ensure youth employability, increased participation in governance by different gender and long-lasting development in Somalia.

During the course of its implementation, SETS II paid special attention to provision of education training opportunities for girls and women, and towards ensuring that all learners of school going age have the knowledge, skills, capabilities, and rights they need to enjoy a life in dignity, to be fully engaged in society as responsible and productive adults, and to contribute to the social, economic and environmental well-being of their communities.

The overall objective of SETS II was: Improve the educational attainment and employability of relevant target groups in Somalia; whilethespecific objectivewas Participation in and governance, quality, relevance and inclusiveness of education and training in Somalia are improved. SETS II was also proactively coordinating with other humanitarianactors and donors to ensure successful implementation and maximum impact of planned activities, and this was achieved through three necessary and sufficient Results:

1. Result 1: Increased access to equitable quality primary and secondary education.
2. Result 2: Increased participation of youth and adults in TVET linked to employment and economic opportunities; and
3. Result 3: Enhanced capacity of public institutions to lead, monitor and manage the education and training system.

1.1 Targeted Primary Beneficiaries

Target beneficiaries included primary/secondary school learners/students and teachers, parents, Community Education Committee (CEC) members and the Community Skills Development Centres (CSDC). Additionally, there were interventions aimed at benefiting youths in TVET and Professional Technical Secondary Schools, those attending Non-Formal Education (NFE), Alternative Basic Education (ABE) and learners with special needs. Overall, there was also a keenness to target children who are returnees, Internally Displaced Persons (IDPs) and/or pastoralists. Based on lessons learned from previous projects and forging strong synergies and collaboration with other ongoing interventions within the project regions, SETS II aimed to ensure that equal education opportunities are provided to all.

1.2 Implementing Partners

The project was implemented by ADRA as the lead agency, in partnership with the Somalia Federal Ministry of Education, Culture and Higher Education (MoECHE), State Ministries of Education of: Galmudug, South West, Hirshabelle, Jubbaland and Banadir Regional Administration; as well as CISP and Alight who were implementing agencies.

1. Evaluation of SETS II Project

2.1 Purpose and Objectives of the Evaluation

These Terms of Reference are for the End of Project Evaluation for the Strengthening Education and Training in Somalia (SETS II) Project. The overall purpose of the evaluation is to assess the level of attainment of targets, results and objectives. The evaluation will contribute to shared learning and provide accountability to partners, beneficiaries and donors. The evaluation will come up with findings, lessons learnt and recommendations which will be shared with key stakeholders of the project and used by the implementing agencies to guide and inform future similar projects and programs.

The evaluation will assess the performance of the project against key parameters including the project’s relevance, effectiveness, efficiency, sustainability, timelines of activity implementation, and its strengths and weaknesses. In addition, the strengths, weaknesses and challenges during implementation will be analyzed with a view to extract lessons and best practices for the future.

1. Objectives of the Evaluation

The final evaluation will provide the decision-makers in the Government, the European Union and the wider public with sufficient information to:

1. Make an overall independent assessment about the past performance of the project/programme, paying particularly attention to the impact of the project actions against its objectives.
2. Identify key lessons and to propose practical recommendations for follow-up actions.

The evaluation will be based on the five evaluation criteria endorsed by the OECD-DAC (relevance, effectiveness, efficiency, sustainability and impact), and to the EC-specific evaluation criteria (EC added value and coherence). The evaluation will also assess the integration and impact of cross cutting issues in the project. The consultants are required to use their professional judgment and experience to review all relevant factors and to bring these to the attention of the Government and European Commission.

The key parameters to be assessed by the evaluation are:

1. Relevance: describes how well a project addresses a real problem of the beneficiaries and how well it matches the development policies and strategic objectives of the country/region.
2. Efficiency: stands for how well the inputs are transformed into output and outcomes.
3. Effectiveness: measures the degree to which the project’s outputs have provided benefits and contributed to the project purpose.
4. Impact: describes how and to which degree the project has contributed to the solution of the problem and to the achievement of the overall objective. While actual impact can only be measured ex post, the evaluation should nevertheless scrutinize the impact prospects, i.e. the project’s likely contribution to the project’s overall objective.
5. Sustainability: introduces a time dimension into the monitoring. It measures to the likelihood of a continuation in the stream of benefits produced by the project after the period of external support has ended.
6. Inclusion: Assess the extent to which marginalized groups have been reached by the project, in particular girls, IDPs, returnees, refugees, people living with disabilities and those from minority clans among other marginalised groups.
7. Role of federal member states: The roles played by the states of Galmudug, Jubbaland, Hirshabelle, South West State and Banadir Regional Administration during the project implementation period, including successes, lessons learnt and gaps identified per State. The support provided to each state also needs to be evaluated.
8. Mutual reinforcement (coherence): assesses the extent to which activities undertaken allow the European Union to achieve its development policy objectives without internal contradiction or without contradiction with other Community policies, i.e. the extent to which they complement partner country’s policies and other donors’ interventions.
9. EU value added: reviews the connection to the interventions of other programs supported by the European Union, and the extent to which the intervention is creating actual synergy (or duplication) with the intervention of other EU supported programmes/projects.
10. Scope of the evaluation

The evaluation will cover the accomplishment of all the expected results as outlined in the project document and detailed in the annual work plans during the period of implementation All the project result areas, outcomes and activities stated in the logical framework matrix of the project will be assessed.

The evaluation team will select sample districts, schools, and communities from the project regions in consultation with relevant stakeholders and using statistically acceptable parameters, and ensure that the sample selected is representative of the geographical coverage of the project as well as the target beneficiaries.

Specific Tasks: The specific tasks of the evaluator will be to:

1. Develop evaluation framework and methodology and refine these with the project team.
2. Develop a sampling frame that will be discussed and adopted by the evaluation team
3. Develop the evaluation plan
4. Undertake extensive document review to familiarize with the project
5. Assess the capacity development interventions the project has undertaken with the MoECHE, State level MoEs and targeted schools
6. Critically assess and evaluate the 5 model schools and 3 TVET centres constructed by the project, looking at the key parameters of: achievement of targets, outcomes and lessons learned.
7. Review the progress in enrolment of learners in primary school, non-formal education and TVET as a result of the project in the target regions.
8. Assess the contribution to pedagogical methods for young learners and adult learners as a result of the project.
9. Conduct field visits, focus group discussions and interviews with relevant beneficiaries, partners and project team members to collect information on various parameters of the evaluation.
10. Prepare the evaluation draft report and present the findings in workshops in Mogadishu, Somalia, for key national and international agencies operating in these locations;
11. Prepare and submit final report to the Lead Agency – ADRA in soft and hard copies.
12. Present the findings of the evaluation to the Education Sector Committee Members in Mogadishu, Somalia.

13. Evaluation Approach and Methodology

The evaluation approach should be developed and implemented in three main phases: An Inception Phase, a Field Investigation Phase and a Synthesis and Feedback Phase.

1. Inception Phase

This phase will involve a thorough and systematic review of the relevant programming documents related to the SETS II project, as well as documents shaping the wider strategy/policy framework. This will provide the development context of the project. The consultant will then analyze the logical framework in order to come up withissues or evaluation questions relevant to the project. This stage will also involve the preparation of the workplan for the evaluation complete with a time schedule. Other tasks to be undertaken during the inception phase will include:

• Present the indicative methodology to the overall assessment of the project/programme.
• Interview the project management and European Union Program Managers (if relevant)
• Present each evaluation question stating the information already gathered and provide a first partial answer to the question, identify the issues still to be covered and the assumptions still to be tested, and describe a full method to answer the question.
• Identify and present the list of tools to be applied in the Field Phase;
• Prepare and submit a detailed work plan with an indicative list of people to be interviewed, surveys to be undertaken, dates of visit, itinerary, and name of team members in charge.
• Define issues and gaps requiring further analysis;
• Prepare and share inception report of not more than 10 pages; and
• List all preparatory steps already taken for the Field Phase.

3.2 Field Investigation Phase

The Field Phase should start upon approval of the inception Report by the evaluation managers (Program Director, Project Manager and M&E focal point). The consultant will:

• Undertake field mission applying the plan developed during the Inception Phase. This plan has to be applied in a way that is flexible enough to accommodate for any last-minute difficulties in the field. If any significant deviation from the agreed work plan or schedule is perceived as creating a risk for the quality of the evaluation, these should be immediately discussed with the evaluation managers.
• Hold a briefing meeting with project management team in the first days of the field phase.
• Undertake school visits and stakeholder interviews including relevant education authorities.
• Ensure adequate contact and consultation with, and involvement of, the different stakeholders; working closely with the relevant government authorities and agencies during their entire assignment. Use the most reliable and appropriate sources of information and will harmonize data from different sources to allow ready interpretation.
• Summarize its field works at the end of the field phase, discuss the reliability and coverage of data collection, and present its preliminary findings in a meeting with the project management team and relevant MoE administrations.
• Their assessment is objective and balanced, affirmations accurate and verifiable, and recommendations realistic.
• When drafting the report, they will acknowledge clearly where changes in the desired direction are known to be already taking place, in order to avoid misleading readers and causing unnecessary irritation or offence.

3.3 Synthesis and Feedback Phase

This phase is mainly devoted to the preparation of the final draft report. The consultant will make sure that:

The consultant will submit the draft evaluation report to ADRA for review. On the basis of comments expressed by ADRA and consortium members & relevant education authorities, the consultant will amend and revise the draft report. The 2nd draft report will be presented to ADRA management, while a summary of the key findings of the evaluation will be presented to a group of stakeholders in Mogadishu under the auspices of the Education Sector Committee for their synthesis and discussion. On the basis of comments made by participants, the consultant will prepare the final version of the report.

1. Expertise Required

2. The following are the qualification requirements for the evaluation consultant:

• Advanced University Degree in Education or related field.
• A minimum ten (10) years of relevant professional experience in the areas of basic/primary education, teacher development, capacity building of education administrations, education economics and in the definition and implementation of sectoral policies in the economic sector;
• Experience in the evaluation of technical assistance project, preferably, particularly those under the education portfolio;
• Fully conversant with the principles and working methods of project cycle management, EC aid delivery methods. knowledge of the activities of multilateral development donors will be an added advantage;
• Solid knowledge of, and practical experience with gender analysis and planning;
• Full working knowledge of English and excellent report writing skills
• Competence and adequate experience in the use of qualitative and/or quantitative methods of data collection and analysis including: sampling, desegregation of data, structured and semi-structured interviewing, focus group discussions, and observation and triangulation research methods.
• Ability to interpret and analyse complex qualitative and quantitative data, and to present findings and recommendations in a clear and concise way.
• Excellent inter-personal communication skills including experience of facilitation and presentation.
• Ability to work equally well with communities and international organisations.
• Knowledge and sensitivity to political and social contexts of Somalia.

1. Time Frame

The evaluation will be carried out from June to July 2025 (50 working days). The consultant will be required to submit a technical proposal indicating the number of days and rate for the consultancy work with a realistic action plan. As a guideline, about 12 days will be dedicated to the Inception Phase, 24 days to the field study phase and 11 days for the synthesis phase, and 3 days for the presentations at Mogadishu level ESC.

6. Supervision and Management

The overall supervision of the evaluators will rest with ADRA. The consultant will report to the Programmes Director (or a designated manager). The consultant will be expected to work closely with, and interact closely with the implementing agency and MoECHE. The ADRA education project manager based in Somalia will provide day-to-day supervision and support to the consultant.

1. 7. Terms and Conditions

2. Logistics: The consultant’s travel from base to the field and back after the end of the contract (including airport tax) will be covered by ADRA.

3. Professional fee: The consultant will propose the professional fee as part of their financial proposal.

4. Tax and insurance: The consultants shall be responsible for their income tax and/or insurance during the assignment.

5. Code of conduct: The consultant is bound by the principles and conditions of ADRA’s Code of Conduct.

6. A contract will be signed by the consultant/evaluator upon commencement of the evaluation which will detail additional terms and conditions of service, aspects on inputs and deliverables.

7. 8. Application Requirements

8. All expressions of interest should include:

Cover letter: A short (maximum three pages) letter addressing the evaluation criteria.

A detailed curriculum vitae

Technical Proposal (maximum eight pages) interpreting the understanding of the TOR, detailed methodology of executing the task, as well as draft evaluation framework and plan.

Financial Proposal: The financial proposal should provide cost estimates for services rendered including daily consultancy fees excluding: accommodation and living costs; transport cost; stationeries, and supplies needed for data collection; costs related to persons that will take part from consortium partners and government authorities during evaluation process, enrichment workshop.

The financial proposal should also provide cost estimates for services rendered including daily consultancy fees related to the consultant and/or associate consultant who will take part in the final evaluation of the project.