One of Capra International Inc.’s areas of concentration is policy and program evaluation in the public and private sectors of Canada, as well as with international organizations. In that context, Capra has extensive experience with both formative and summative evaluations.
Formative evaluations are also known as implementation evaluations, since they assess, usually at mid-term, the extent to which policies and programs, or initiatives, are being implemented as planned, and which adjustments, if any, may be required.
Summative evaluations are also known as impact evaluations, since they focus on the extent to which expected results have been achieved during a given cycle of operation.
A special variant of both evaluation types is the horizontal evaluation. This type may be very complex, since it involves as client more than one organization. Significant difficulties may arise when trying to achieve a unified understanding among the partners regarding the substance and approach to the evaluation, as well as data sharing (see presentation notes).
All types of evaluation usually involve both secondary and primary research. Secondary research usually comes first and consists of a review of existing documents, files and databases. Primary research follows secondary research. It consists of developing new knowledge by means of several lines of evidence (or data sources), such as key informant interviews, focus groups, case studies, surveys and site visits.
The planning for an evaluation normally consists of the development of a detailed Evaluation Framework, comprising a profile of the policy, program or initiative to be evaluated (including a logic model), a description of the objectives and context of the evaluation, the issues, questions and indicators to be addressed, data analysis and reporting the stakeholders involved, the deliverables expected, the corresponding timeline, the human resources involved to execute the evaluation, as well as the overall budget.
To achieve a high degree of quality, various organizations have developed quality control standards, and or norms, for the execution of evaluations. Capra’s standards can be accesses by following this link.
Some evaluation assignments address only certain parts, such as case studies, surveys, etc. Others focus on preparatory activities, such as performance measurement. The latter concerns the definition of performance indicators and the corresponding data collection. This activity is typically performed by management on an ongoing basis. When this task is performed by a third party on behalf of a client, it is often referred to as “monitoring”.
In Canada, evaluations within the federal government must be executed in accordance with the Treasury Board of Canada evaluation policy.
Capra routinely offers evaluation and performance measurement services in a multi-language environment. Besides English and French, Capra has offered services in: Spanish, Mandarin, Cantonese, Vietnamese, Tagalog, Dari, Hindi, Punjabi, Gujarati, Somali, Arabic, as well as the Canadian indigenous languages of Ojibwe and Inuktitut. Other languages are also available upon request.
Partial List of Current and Recent Evaluation Projects:
Formative evaluations are also known as implementation evaluations, since they assess, usually at mid-term, the extent to which policies and programs, or initiatives, are being implemented as planned, and which adjustments, if any, may be required.
Summative evaluations are also known as impact evaluations, since they focus on the extent to which expected results have been achieved during a given cycle of operation.
A special variant of both evaluation types is the horizontal evaluation. This type may be very complex, since it involves as client more than one organization. Significant difficulties may arise when trying to achieve a unified understanding among the partners regarding the substance and approach to the evaluation, as well as data sharing (see presentation notes).
All types of evaluation usually involve both secondary and primary research. Secondary research usually comes first and consists of a review of existing documents, files and databases. Primary research follows secondary research. It consists of developing new knowledge by means of several lines of evidence (or data sources), such as key informant interviews, focus groups, case studies, surveys and site visits.
The planning for an evaluation normally consists of the development of a detailed Evaluation Framework, comprising a profile of the policy, program or initiative to be evaluated (including a logic model), a description of the objectives and context of the evaluation, the issues, questions and indicators to be addressed, data analysis and reporting the stakeholders involved, the deliverables expected, the corresponding timeline, the human resources involved to execute the evaluation, as well as the overall budget.
To achieve a high degree of quality, various organizations have developed quality control standards, and or norms, for the execution of evaluations. Capra’s standards can be accesses by following this link.
Some evaluation assignments address only certain parts, such as case studies, surveys, etc. Others focus on preparatory activities, such as performance measurement. The latter concerns the definition of performance indicators and the corresponding data collection. This activity is typically performed by management on an ongoing basis. When this task is performed by a third party on behalf of a client, it is often referred to as “monitoring”.
In Canada, evaluations within the federal government must be executed in accordance with the Treasury Board of Canada evaluation policy.
Several Capra evaluators hold membership in the Canadian Evaluation Society (CES) and in the International Development Evaluation Association (IDEAS). Capra received international recognition by the Independent Evaluation Group (IEG) of The World Bank, which recognized Capra International Inc. as an evaluation provider in its newSourcebook of Indicative Principles and Standards for Evaluating Global and Regional Partnership Programs (2007).
Capra routinely offers evaluation and performance measurement services in a multi-language environment. Besides English and French, Capra has offered services in: Spanish, Mandarin, Cantonese, Vietnamese, Tagalog, Dari, Hindi, Punjabi, Gujarati, Somali, Arabic, as well as the Canadian indigenous languages of Ojibwe and Inuktitut. Other languages are also available upon request.
- Finance Canada. 10-yr Evaluation of Canada’s Anti-Money Laundering and Anti-Terrorist Financing Regime (awarded in February 2010).
- Foreign Affairs and International Trade and Canadian International Development Agency. Canada Fund for Africa Mid-term Review (awarded in October 2009).
- Human Resources and Skills Development. Summative Evaluation of the International Trade and Labour Program (awarded in July 2009).
- Industry Canada, Office of Consumer Affairs. Evaluation of the Contribution Program for the Non-Profit Consumer and Voluntary Organizations (awarded in April 2009).
- Human Resources and Skills Development. Employer Survey for the Sector Council Program (2009).
- Human Resources and Skills Development. Development of an Evaluation Framework for the Federal Elder Abuse Initiative (awarded in Februry2009).
- SNV Netherlands Development Organisation. Evaluation of the Honduras Country Program (2008).
- Suriname Ministry of Trade and Industry (MTI). Evaluation of the Trade Sector Support Program (TSSP) (awarded in October 2008, continuing).
- Health Canada. Summative Evaluation of the Contribution Program to Improve Access to Health Services for Official Language Minority Communities (awarded in January 2008).
- Public Safety Canada. Summative Evaluation of the National Security Capacity Initiative (awarded in January 2008).
- Department of Canadian Heritage. Evaluation of the Trade Routes Program (2007-2008)
- Service Canada. Evaluation of Multi-language Service Initiative (included client survey in two Canadian Aboriginal and 10 international languages) (2007-2008).
- Canadian Heritage. Formative Evaluation of the Law Enforcement Aboriginal and Diversity Network (2007).
- Department of Natural Resources (Canada). Evaluation of the North American Forest Commission (Canada, Mexico, USA) (2006).
- Department of Natural Resources (Canada). Development of a Geopolitical Analysis Framework for International Relations in Natural Resources and its application in China, India and Brazil (2004).
- World Trade Organization, The World Bank, International Monetary Fund, International Trade Centre, UNCTAD, UNDP. Evaluation of the Integrated Framework for Trade-Related Technical Assistance to Least Developed Countries (Haiti, Mauritania, Senegal, Guinea, The Gambia, Mali, Uganda, Malawi, Bangladesh, Nepal, Cambodia, Yemen, Tanzania, Lesotho, Burundi, Djibouti, Ethiopia, Eritrea, Madagascar) (2003).