Terms of Reference

End-Term Evaluation for SIDA-Funded

“Promoting Inclusion and Combating Discrimination” Program

Office NDI-Ukraine

Program Promoting inclusion and combating discrimination

Type of Contract and position Consultant

Location Kyiv, Ukraine

PURPOSE

This document outlines the scope of work for a consultant to conduct a program assessment for NDI in Ukraine. The consultant will work in coordination with the NDI’s team in Ukraine. The work will include the following:

1. Develop a matrix that includes evaluation questions and a detailed description of the proposed data collection and analysis methodology and sampling strategy.

2. Develop instrumentation and data collection and analysis protocols.

3. Lead the data collection for the virtual and in-person fieldwork.

4. Draft and finalize the program evaluation report.

BACKGROUND

The National Democratic Institute (NDI) has worked in Ukraine since 1992 to support Ukrainians’ aspirations for a more democratic future. Approaches have included bolstering civil society and political development, strengthening parliament, supporting women’s political participation, promoting public consultation, gender equality and inclusion, combatting discrimination, and conducting public opinion research.

NDI launched its program aimed at promoting women’s political participation in Ukraine in 2010. At that time, gender equality was a marginalized issue in political discourse. In 2011 NDI helped establish the Equal Opportunities Caucus (EOC) in the parliament, which since then has put forward and advocated for numerous important legislative changes in support of gender equality. Due to the tireless efforts of gender advocates, attitudes toward gender equality have changed significantly since the 2014 Revolution of Dignity. According to NDI’s 2023 poll, a historic number - 79 percent of Ukrainians - want more equal representation of women and men in political life (Opportunities and Challenges Facing Ukraine’s Democratic Transition. Nationwide telephone survey). 

In 2017, for the first time in its history, the Ukrainian government named gender equality as one of its priorities and formally designated the Vice Prime Minister for European and Euro-Atlantic Integration of Ukraine (hereinafter - VPM Office) to be responsible for overseeing national gender equality policy. Same year the Office of the Government Commissioner for Gender Equality Policy (hereinafter - OGE) was established.

Since that time Ukraine made significant progress in the development and implementation of gender equality policy at the national level. In 2020 for the first time in Ukrainian history, the election legislation includes an enforceable gender quota for party lists, and in 2021 Ukraine became a member of the Biarritz Partnership for Gender Equality. A set of national strategic documents in support of gender equality was adopted, including but not limited to State Strategy for Ensuring Equal Rights and Opportunities for Women and Men till 2030 (2022), Strategy for Implementation of Gender Equality in Education until 2030 (2022) and Strategy for Bridging the Gender Pay Gap for the period up to 2030 (2023). Furthermore, in 2022 as a result of the long-term efforts of the stakeholders and gender equality advocates Ukraine ratified the Council of Europe Convention on Preventing and Combating Violence Against Women and Domestic Violence (Istanbul Convention).

At the same time, the full-scale war of Russia against Ukraine has had a significant impact on gender equality in Ukraine introducing new challenges, worsening the socioeconomic well-being of women and reinforcing gender stereotypes. The latter is supplemented by Russian propaganda focusing on “traditional” role of women and anti-LGBTQ+ rhetorics.

NDI’s LGBTQ+ programming is evidence-based. Public opinion research has shown that many Ukrainians have an unformed opinion about LGBTQ+, and awareness-raising about inclusion and tolerance has the potential to change public opinion. As of November 2023, according to an NDI public opinion survey, 72 percent of Ukrainians agree that LGBTQ+ people should have the same rights as others, an increase in sentiment since the beginning of the full-scale Russian war against Ukraine. At the same time, there still are many obstacles on the way to truly equal participation of LGBTQ+ individuals in social, economic, and political life in Ukraine. During the full-scale Russian war against Ukraine, LGBTQ+ people have encountered increased negative impacts in social, legal, and economic areas, as well as a continued lack of legal and substantive protection against hate crimes.

EVALUATION OBJECTIVES

The overarching goal of the evaluation would be to assess NDI’s SIDA-funded program in achieving its results using the Organization for Economic Co-operation and Development/Development Assistance Committee (OECD/DAC) criteria as a guide in measuring the impact of the program, and providing lessons learned and recommendations to inform the design of future programming. However, criteria can be adjusted in discussion with the evaluator or evaluation team. The OECD DAC criteria focus for this specific program are as follows:

1. The effectiveness of the program in achieving its intermediate results, as well as any unintended results that emerged after project startup;

2. The efficiency of the program in producing outputs—qualitative and quantitative—in relation to the inputs;

3. The relevance of the program to the original and emerging development problems and its proposed critical assumptions and risks, and adaptation strategies; and

4. The potential impact of the program results.

Other specific goals of the evaluation include the following:

1. Analyzing the overall gender and LGBTQ+ context in Ukraine and the role of NDI’s SIDA-funded programming;

2. Analyzing the outcomes and objectives of the program and assess their effectiveness and sustainability;

3. Documenting success stories produced by the program;

4. Documenting program contribution in relation to complementary programs and added value of the program.

5. Documenting best practices and lessons learned produced by the program; and

6. Identifying new programmatic windows of opportunity and based on that (and on lessons learned and recommendations) provide recommendations to inform the design of future programming.

TIMELINE

The deadline for the report submission is May 27, 2024.

The detailed timeline will be discussed and confirmed with the selected candidate.

EVALUATION PROCESS

The expert will be responsible for refining the proposed evaluation questions in close collaboration with NDI. The expert is expected to develop a robust evaluation design and methodology to respond to the agreed evaluation questions.

The expert is expected to conduct a desk review of current programming documents, interview NDI program staff, and conduct key informant interviews with project partners and beneficiaries, including women and LGBTQ+ prospective political candidates or activists and elected officials, and representatives of CSOs and media.

The evaluator will discuss and finalize the methodology together with NDI and its DMEL staff. NDI program evaluation tools are designed separately for each office and context and can be a mix of:

● Desk review and background research of program documents and reports from other

organizational sources

● Quantitative instruments (surveys, etc.)

● Qualitative instruments, key informant interviews with project partners and participants, focus group discussions, etc.

● Outcome harvesting, outcome mapping, or other relevant complexity-aware evaluation methodology or techniques The evaluators will be expected to develop an evaluation strategy and delineate tasks necessary for the successful completion of the evaluation, including:

● Desk review of program documents and partner outputs, including NDI donor reports and tools and reports produced by partners;

● Design of the overall evaluation strategy and methodology, including identification of appropriate data collection and analysis methods, development of evaluation instruments and questionnaires, and establishment of protocols for data collection and analysis;

● Data collection and analysis, likely including a review of project reports, partner outputs, data collected through ongoing program monitoring, interviews with key informants and NDI field office program team;

● Data analysis against program results;

● Submission of draft report for comments by NDI and SIDA;

● Submission of final evaluation report.

DELIVERABLES

1. Informed consent documentation.

2. Data confidentiality plan.

3. Data collection, analysis instruments, and protocols, including the design of interview

questionnaires.

4. Raw and processed data (recorded interviews will be primary data collected).

5. Draft report.

6. Final report.

REPORTING

The final report is expected to provide a comprehensive response to all evaluation questions outlined above supported with evidence gathered during the evaluation exercise. The report should be in English.

The final report should include:

1. Executive summary of findings, conclusions, and recommendations.

2. Introduction and background to the evaluation.

3. Evaluation questions, methodologies and approach, timeline, scope, constraints, and limitations.

4. Description of the project objectives evaluated.

5. Data collection and analysis.

6. Key findings and conclusions supported with evidence.

7. Recommendations for future programming to be implemented by NDI.

Annexes should include:

● TOR, references, list of meetings and documents used for evaluation.

COMPETENCIES

● Experience in designing, implementing and managing program evaluations of democracy assistance interventions. An understanding of experimental research design is a plus.

● Experience using innovative monitoring and data collection methods a plus (e.g., outcome harvesting, complexity aware monitoring, most significant change, social network analysis)

● Qualitative and Quantitative data analysis skills. Experience using both qualitative and quantitative analysis software a plus (e.g., SPSS, STATA, DEDOOSE, AtlasTi, Nvivo)

● Strong communication and coordination skills and experience working with a cross section of organizational stakeholders.

● Fluency in English is required.

● Ability to work remotely as part of a team, as well as independently.

SUBMISSION

Suitable and qualified candidates are requested to send a CV and motivation letter as one PDF file in English to the following email address: [email protected]. Please indicate the position - Program Assessment Consultant - you are applying for in the subject line of the email. Your CV and motivation letter should demonstrate your interest and qualifications for this position.

The deadline for the submission of the required documents is the end of the day of April 8, 2024. Only shortlisted candidates will be contacted for the written assessment, and finalists only will be invited for an interview