Evaluation of PPSC Activities resulting from Funding to Implement the Canadian Victims Bill of Rights - Final Evaluation Report - February 2021

Internal Audit and Evaluation Division

Approved by the Director of Public Prosecutions on February 22, 2021.

© Her Majesty the Queen in Right of Canada, 2021.

Cat. No. J79-10/2021E-PDF
ISBN: 978-0-660-39190-8

Contents

1. Introduction

The evaluation of the Public Prosecution Service of Canada's (PPSC) Crown Witness Coordinator (CWC) program was undertaken as part of the implementation of the Canadian Victims Bill of Rights (CVBR). The purpose of the CWC program evaluation was to assess the program's increased capacity through predetermined performance indicators by examining both qualitative and quantitative data. The evaluation covers the time period from January 2015 to January 2020.

2. Background

CWC Program Description

The CWC program is unique to Canada's three northern territories: Yukon, Northwest Territories, and Nunavut (total population of 108,973 in 76 communities). Fourteen (14) CWCs work closely with the forty (40) Crown prosecutors in the three (3) northern PPSC offices. They locate, support, and prepare crime victims and witnesses for their role in the prosecution of crimes. CWCs act as liaisons between the Crown prosecutor and victims/witnesses, keeping both parties apprised of developments and situations, which impact their cases. CWCs encourage the use of Victim Impact Statements, assist in the arrangement of testimonial aids, attend court with victims/witnesses, and follow up with them as necessary. CWCs also arrange referrals to other supportive services. They work with the resident courts in Whitehorse, Yellowknife, and Iqaluit and travel with the circuit court to 57 small territorial communities (13 communities in Yukon, 21 communities in Northwest Territories, and 23 communities in Nunavut).

In the course of their work CWCs, and other criminal justice system professionals, must deal with challenges and circumstances specific to northern Canada: the highest rates of violent crime in Canada; chronically traumatized victims and witnesses; extensive travel to small isolated communities in difficult weather; a heavy workload with court imposed deadlines; a variety of cultural traditions, beliefs, and languages; a lack of community based victims' assistance resources; and widespread lack of understanding, and wariness, of the criminal justice system.

Prior to 2015, the Northwest Territories and Nunavut CWCs were unable to meet the demand for their services due to increasing crime rates, staffing shortfalls, and the growing number of victims and witnesses spread over many isolated, small communities.

In 2014, the PPSC submitted a request to Treasury Board for funding support of four new CWC positions, one each in Whitehorse and Yellowknife, and two in Iqaluit. The PPSC's report to Treasury Board indicated that with the additional resources they would be able to effectively implement the CVBR. The PPSC also drafted five indicators that would be verified as part of a five-year evaluation (2015-2020). The following performance indicators were as follows:
1) Number of additional CWCs on strength by territory and fiscal year; 2) Number and percentage (%) of court circuit parties that included a CWC, where required; 3) Number of victims assisted by CWCs, by territory and type of file (with type of file determined by most serious offence) by fiscal year; 4) Nature of prosecution-related processes or procedures to promote compliance with relevant provisions of the (CVBR); and 5) Number and nature of training sessions offered on the provisions and implications of the CVBR to CWCs, Crown Prosecutors and/or other justice officials.

The PPSC committed to the evaluation as part of the Treasury Board Submission.

3. Evaluation Objective

The objective of the CVBR evaluation is to measure the impact of the additional funding provided to the CWC program as part of the implementation of the CVBR.

In addition, the evaluation also sought to understand how the effectiveness of the program improved over the evaluation period and how it might be improved in both the short and long term.

4. Methodology

This section provides a brief description of the methodology used to measure the impact of the additional funding provided to the CWC program as part of the implementation of the CVBR.

4.1. Evaluation Approach

The evaluation assessed the extent to which the additional four CWCs contributed to the Program's expected outcomes.

In order to successfully address the evaluation issue and questions covered by the evaluation, some existing data and information collected throughout the Program's implementation and ongoing management have been used. Additional evidence was collected through a document review and key informant interviews.

Seven interviews were conducted with northern criminal justice professionals: three CWCs, three CWC supervisors, and one senior official of the PPSC, Northern Affairs. See appendix 1 for key informant interview questionnaires.

Due to privacy concerns, victims and witnesses were not interviewed during the evaluation process. However, to the extent possible, the perceptions and experiences of victims and witnesses involved with the northern criminal justice system, and therefore the CWC program, have been gathered from the other internal reports or documents.

Both the data collection and analysis conducted as part of this evaluation align with the overall framework provided by the federal government's Policy on Results which expects evaluation to support ongoing accountability, inform government decisions on resource allocation, and support the ongoing management and improvement of the Program.

All research activities undertaken as part of this evaluation were administered in accordance with normal practices in the field of program evaluation.

4.2. Methodological Limitations

Broad outcomes of the Program

The main limitation was that the broad nature of the intended outcomes of the Program made it difficult to measure whether these had been achieved. For example, all CWC respondents believed there was insufficient evidence to determine whether the increased number of CWCs had contributed to the number and percentage (%) of victims which were assisted by a CWC but all CWCs interviewed believed that more time was given to victims of less serious crimes.

Similar responses from respondents

The interviews with key informants have the possibility of self-reported response bias. This occurs when individuals are reporting on their own activities and answer questions with the desire to affect outcomes. In order to mitigate these limitations, triangulation was used to check findings against other sources and perspectives. The evaluation was able to demonstrate how the Program's additional CWCs were used and assessed results in general terms, by using evidence from the document review, and the views expressed by key informants.

Limited amount of Program-related statistics for the evaluation period

Unfortunately, statistics related to the indicators were limited and covered shorter periods than the evaluation period. Having these statistics may have provided more evidence of the increased capacity of the Program.

Data collection was requested of the territories in reference to two performance indicators: 1) Number and percentage (%) of court circuit parties that included a CWC, where required and 2) Number of victims assisted by CWCs, by territory and type of file (with type of file determined by most serious offence) by fiscal year.

5. Findings

5.1. Evaluation Performance: Effectiveness and Enhanced Capacity

Continued improvement of data collection

During the evaluation it became clear that the Northern Regions did not have consistent CWC program data collection practices. While iCase data collection in each northern PPSC office routinely tracks the number of Crown cases and the number of victims and witnesses, there is agreement among evaluation respondents that the data collection process could continue to be improved. For example, data collection only started in the first fiscal quarter of 2017 when it should have started with the CVBR implementation in 2015. In addition, some data was only collected every second month when it should have been collected on a monthly basis.

Two of three CWCs mentioned that there is more paperwork, tracking of victims, and data collected since the implementation of the CVBR.

Increased CWC time spent with victims

All CWC respondents stated that the new CWCs helped them meet the CVBR requirements and increase the capacity of the program primarily because victims receive more time with a CWC regardless of the severity of the crime. One respondent added there is more time to prepare for upcoming circuits.

All CWC supervisors agreed that the CVBR resulted in more involvement, interaction, and administrative work. They mentioned it was a great improvement, even if locating victims is still proving to be difficult.

The PPSC official stated that CWCs need to build relationships and trust with the victims and that is the key component to the enhanced capacity of the Program.

Human resources issues

CWC supervisors raised concerns regarding staffing, and more specifically, about the difficulty of attracting qualified CWCs with a specific skill set (i.e. indigenous language). According to PPSC HR records it has taken time to hire new CWCs. Contributing to the delays for Nunavut is the limited number of qualified candidates who speak an indigenous language. The second new CWC in Nunavut was hired in October 2016 but the position was created in December 2015.

Respondents also mentioned that CWCs have a higher than average absenteeism rate because of the emotionally taxing nature of the job. Therefore, even if the region is operating at full capacity the high absenteeism rate affects the performance of a fully staffed unit.

It is important to note that there has been turnover of the CWC supervisor in two of the three territories. One CWC was acting supervisor for two years and currently two of the three territories do not have someone in the supervisor position; they are currently supervised by either a Chief Federal Prosecutor or the General Counsel – Legal Operations.

5.2. Performance Indicators

5.2.1. Number of additional CWCs on strength by territory and fiscal year

Prior to the evaluation period, the Yukon had four CWC positions and the additional TB funding provided them the opportunity to increase their overall CWC unit by staffing a new CWC in April 2015. They currently have five CWC positions.

The Northwest Territories previously had six CWC positions and the additional TB funding provided them the opportunity to increase their overall CWC unit by staffing a new CWC in April 2016. They currently have seven CWC positions.

Nunavut previously had six CWC positions and the additional TB funding provided them the opportunity to increase their overall CWC unit by staffing two new CWC. One position was staffed in September 2015 and the second one was created in December 2015 but only staffed in October 2016. They currently have eight CWC positions.

5.2.2. The number and percentage (%) of court circuit parties that included a CWC, where required, by territory and by fiscal year quarter

It is important to note that the CVBR data collection only started in April 1, 2017. Therefore, the territories were only able to provide statistics from April 1, 2017 to January 1, 2020.

The following tables show the quarterly and yearly averages for each territory. The Yukon has an average annual coverage rate of 100%. The Northwest Territories lowest annual average coverage rate was 81% and Nunavut's lowest annual average coverage rate was 97%.

Overall, all territories had a high number of court circuit parties that included a CWC.

Yukon
    FY 2017/18 FY 2018/19 FY 2019/20
    Q1 Q2 Q3 Q4 Ave Q1 Q2 Q3 Q4 Ave Q1 Q2 Q3 Q4 Ave
Number and percentage (%)
of court circuit parties
with CWC
  18/18 20/20 16/16 20/20  
100
19/19 21/21 17/17 22/22
100
17/17 20/20 19/ 19  
N/A
 
100
% 100 100 100 100 100 100 100 100 100 100 100
Northwest Territories
    FY 2017/18 FY 2018/19 FY 2019/20
    Q1 Q2 Q3 Q4 Ave Q1 Q2 Q3 Q4 Ave Q1 Q2 Q3 Q4 Ave
Number and percentage (%)
of court circuit parties
with CWC
  25/ 27 20/23 22/ 22 22/ 24  
93
26/ 26 19/ 25 20/ 25 21/ 24
86
25/ 32 19/ 22 15/ 19  
N/A
 
81
% 93 87 100 92 100 76 80 88 78 86 79
Nunavut
    FY 2017/18 FY 2018/19 FY 2019/20
    Q1 Q2 Q3 Q4 Ave Q1 Q2 Q3 Q4 Ave Q1 Q2 Q3 Q4 Ave
Number and percentage (%)
of court circuit parties
with CWC
  24/ 24 21/ 22 24/25 20/ 20  
98
24/ 24 18/ 19 20/ 20 23/ 25
97
24/ 24 18/ 18 22/ 22  
N/A
 
100
% 100 95 96 100 100 95 100 92 100 100 100

5.2.3. The number of victims assisted by CWCs, by territory and type of file (with type of file determined by most serious offence) by fiscal year

An additional limitation to the analysis was that the following data was only collected every two months. Therefore, only seven months of data was available for each fiscal year.

Unfortunately, this performance indicator did not provide any insight on the enhanced capacity that additional CWCs had on the number of victims assisted by CWCs. For example, the following tables identifies the number of victims assisted by CWCs for the five types of serious offences, but we do not have the total number of victims who were assisted by CWCs.

Yukon
Type FY 2017/18 FY 2018/19 FY 2019/20
Q11 Q22 Q33 Q44 Q11 Q22 Q33 Q44 Q11 Q22 Q33 Q4
Homicide 0 0 0 1 1 0 0 0 0 0 0 N/A
Assault 41 9 19 33 4 31 32 50 23 21 12
Property 5 3 10 5 5 25 9 7 5 5 1
Sexualized 4 5 4 3 5 2 4 5 4 4 0
Driving 0 0 0 0 0 1 0 0 0 0 0
Northwest Territories
Type FY 2017/18 FY 2018/19 FY 2019/20
Q11 Q22 Q33 Q44 Q11 Q22 Q33 Q44 Q11 Q22 Q33 Q4
Homicide 0 0 0 0 0 2 3 0 1 1 0 N/A
Assault 113 88 129 62 27 58 43 151 63 86 43
Property 46 29 18 24 11 11 20 39 10 10 6
Sexualized 13 25 27 13 0 10 7 14 7 12 6
Driving 0 3 0 0 0 1 0 0 0 0 0
Notes: 1 Data is only available for April and June 2017 1 Data is only available for May 2018 1 Data is only available for May 2019
2 Data is only available for August 2017 2 Data is only available for July and September 2018 2 Data is only available for July and September 2019
3 Data is only available for October and December 2017 3 Data is only available for November 2018 3 Data is only available for November 2019
4 Data is only available for January and March 2018 4 Data is only available for January and March 2019
Nunavut
Type FY 2017/18 FY 2018/19 FY 2019/20
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
Homicide 0 0 0 0 2 0 0 1 0 0 0 N/A
Assault 104 50 90 51 101 49 92 67 77 42 44
Property 11 4 15 6 4 13 15 6 6 9 8
Sexualized 19 13 21 10 13 11 18 6 8 6 4
Driving 0 2 0 0 0 0 0 0 0 0 0

5.2.4. The nature of prosecution-related processes or procedures to promote compliance with relevant provisions of the CVBR

All regional offices received copies of pamphlets (Your Rights as a Victim of Crime) translated into the various regional languages. CWCs were required to make contact with all victims of crimes and covered related provisions of the CVBR. In addition, Performance Management Agreements for CWCs include coverage of the CVBR with all victims.

Information regarding the CVBR and the rights of victims is also mentioned throughout the CWC training binder.

5.2.5. The number and nature of training sessions offered on the provisions and implications of the CVBR to CWCs, Crown Prosecutors and/or other justice officials

Training Nunavut Regional Office Northwest Territories Regional Office Yukon Regional Office
Teleconference training sessions held with all CWCs in three separate regional sessions. Introduction to the CVBR - NRO Spring of 2015 attended by 7 CWCs. Introduction to the CVBR - NWTRO Spring of 2015 facilitated and attended by 7 CWCs. Introduction to the CVBR - YRO Spring of 2015 facilitated and attended by 4 CWCs.
In person CVBR overview to Crown Counsel.     Introduction to the CVBR - YRO Crown Spring of 2015 at a litigation committee meeting.
In person CVBR overview to RCMP.     Introduction to the CVBR - M Div RCMP Spring 2015 provided to section heads.
Comprehensive CVBR training to CWCs in person training. In person CVBR training NRO Fall of 2015 facilitated and attended by 6 CWCs. In person CVBR training NRO Fall of 2015 facilitated and attended by 6 CWCs. In person CVBR training NRO Fall of 2015 facilitated and attended by 4 CWCs.
Comprehensive CVBR training - power point presentation. Provided copies of the power point presentation to new CWCs. Power point presentation with all CWCs January 2020.  

6. Conclusion and Recommendations

This section of the report provides a conclusion on the evaluation issue addressed in this report. Where appropriate, recommendations are included.

6.1. Performance

6.1.1. Outcome Achievement

During the period covered by this evaluation, the undefined performance indicators and the CWC Program's data collection inconsistency from all three territories made the intended outcomes of the enhanced capacity of the Program difficult to measure.

6.1.2. Efficiency

All interview respondents mentioned that the additional funding for four more CWCs has provided more coverage and support to victims. Nevertheless, the document review and interviews indicate that efficiencies could be found in better administrative and data collection methods.

6.2. Recommendations

In order to increase the accountability and the ability to assess the performance, of the CWC Program, the following recommendations are submitted:

  1. Management should identify, establish, and report on relevant performance indicators to better assess the effectiveness of the CWC Program.
  2. Management of the CWC Program should advise the CWCs that all CVBR data collection required for the performance indicators should be registered on a monthly basis and added as a mandatory requirement in their training manual. Responsibility should be assigned to the CWC Program Coordinator to ensure that the data is collected.
  3. The PPSC should conduct another CWC Program evaluation within three years.

6.3. Management Action Plans

Recommendation 1: Management should identify, establish, and report on relevant performance indicators to better access the effectiveness of the CWC Program.

Action to be Taken Responsible Individual Date
All three Northern Regions and the Special Advisor on Northern Issues will agree on CVBR-related performance indicators. Final approval of the measures by the Director of Public Prosecutions (DPP) and Deputy Directors of Public Prosecutions. CFPs and Special Advisor on Northern Issues Completed by July 31, 2020
Establish, document, communicate, and implement data entry requirements. CFPs and Special Advisor on Northern Issues Completed by August 31, 2020
Report on performance data on a quarterly basis. The data will be collected by the Crown Witness Program Coordinator (CWPC) who will then report on the results to Special Advisor on Northern Issues Beginning December 31, 2020 and thereafter quarterly

Recommendation 2: Management of the CWC Program should advise the CWCs that all CVBR data collection required for the performance indicators should be registered on a monthly basis and added as a mandatory requirement in their training manual. Responsibility should be assigned to the CWC Program Coordinator to ensure that the data is collected.

Action to be Taken Responsible Individual Date
CWCs to open VRAs directly in iCase and update VRA information to iCase within 2 weeks of return from circuit or completion of court week. CFPs System established by
September 1, 2020
Update the CWC Manual to include monthly CVBR data collection requirements and communicate this to the CWCs. CWPC who then advises Special Advisor on Northern Issues when completed August 31, 2020
Quarterly compliance reporting by region. CWPC who then reports to Special Advisor on Northern Issues Beginning December 31,2020 and thereafter quarterly

Recommendation 3: The PPSC should conduct another CWC Program evaluation within three
years.

For this, no action plan is required. We will seek final agreement to this from the DPP.

Appendix 1: Key Informant Interview questionnaires

Public Prosecution Service of Canada official

Are there alternative ways other than increasing the number of CWCs to better achieve the enhanced capacity of the Program?

To what extent did the ODPP collaborate with police and other concerned criminal justice officials to develop mechanisms to promote compliance with CVRB provisions as they relate to prosecutions?

CWC Supervisors

To what extent did the new CWCs contribute to the enhanced capacity of Program?

Have you seen an improvement in the service provided by the CWCs since more CWCs were hired? Please describe any gaps of services provided and how you intend to address them.

Please describe any challenges, if any, that you encountered during the hiring of additional CWCs?

CWCs

To what extent did the new CWCs contribute to the enhanced capacity of Program?

Have you spent more time with CVBR victims since new CWCs were hired?

Have you seen an improvement in the CWC Program since more CWCs were hired? If yes, can you list certain improvements?

Date modified: