The International Criminal Investigative Training Assistance Program (ICITAP), of the U.S. Department of Justice, recently published a report on Promoting Sustainable Institutional Law Enforcement Development. ICITAP focuses on promoting sustainable institutional law enforcement development (SILED), which the report defines as “providing technical assistance, mentoring, training, and internships that enable host-country law enforcement organizations to improve their capacity and efficiency of operations, their ability to effectively serve citizens, their respect for human rights and human dignity, and their professional standards.” The report is presented in three sections: the first describes ICITAP’s analytical framework for promoting SILED, the second discusses several case studies of their projects, and the final section outlines issues relevant to program monitoring and evaluation.
The report emphasizes the importance of engaging in preliminary steps to effectively design, implement, and evaluate international law enforcement development programs. The report argues that the chances of success in promoting SILED are low if a project is initiated with either a lack of knowledge or preconceived perceptions regarding the needs of host governments, and the report advocates for an approach based on comprehensive assessment and analysis to guide program development. To this end, the report outlines a ten-step process that both increases the likelihood that assistance programs will have a sustainable and positive impact and serves to build trust and rapport between ICITAP and host country stakeholders.
Steps one through four are categorized as the research and assessment phase of the framework and consist of:
- A desk study to discover what data already exists, initiate collaboration with others working on similar issues, and gain a baseline understanding of relevant issues to aid the development of a preassessment measure. In addition to a focus on criminal justice systems, the study should also consider regional “histories, governments, political systems, and existing socioeconomic factors;”
- A preassessment questionnaire to gather additional information from primary stakeholders to foster an efficient and focused on-ground assessment. In addition to providing logistical suggestions for questionnaire distribution, the report recommends that the questions focus on such issues as primary crime threats, strengths and weaknesses of law enforcement entities, the role of the military and their interaction with law enforcement agencies, and the relationship between law enforcement and the public;
- A follow-up discussion with host-country officials, which offers an opportunity for ICITAP to explain their intentions to host-country officials, as well as to discuss any questions or concerns regarding the preassessment questionnaire; and finally
- An on-ground assessment consisting of interviews with stakeholders in the host-country with the goal of building upon information gathered in the previous steps to develop a thorough understanding of criminal justice institutions. Collection of data on management, investigative, and functionary ratios, as well as budget processes, is emphasized, as is the importance of interviewing individuals both within and outside of law enforcement and criminal justice communities.
Steps five through eight represent the analysis phase of the framework and consist of:
- A crime threat analysis to help determine the degree to which “the current priorities, activities, and capabilities of law enforcement meet the demands of citizens and the threats posed by key crime and public safety issues.” Key steps to conducting a crime threat analysis include planning and preparing; collecting and evaluating data; processing, analyzing and reviewing information; and making recommendations;
- A job task analysis to gain an understanding of the skills and responsibilities required to be successful in a given law enforcement role. Compiling a list of job tasks, establishing the importance of these tasks, and identifying the capabilities essential to successfully performing these tasks are identified as three steps of a job task analysis;
- An institutional development analysis to evaluate institutional governance and identify any major gaps or deficiencies in policies, procedures, processes, or practices that hinder basic operations. Institution structure, policy development, long-term planning and execution, recruitment and training, and appropriation of financial, human, and operational resources are all areas that should be covered in the analysis. Key steps of this analysis include determining the structure, as well as current policies, procedures, processes, and practices of the institution; finding discrepancies between how such policies are written and how they are implemented, foster decision makers in creation or modification of policies, and continued monitoring; and
- A training needs analysis to “assess law enforcement’s current capabilities, identify areas and populations where training is needed, and determine the type of training required.” The key steps to conducting this analysis are identifying data sources, collecting and analyzing data, and proving recommendations.
The final steps, 9) develop program design and training curricula, and 10) develop a monitoring and evaluation plan, should be completed based on the information gathered in steps one through eight.
Several case studies are illustrative of ICITAP’s success in promoting SILED across various contexts. The report specifically discusses ICITAP’s Police Precinct (Polsek) Reorganization Program in Indonesia, the Public Order Management Program in Bosnia and Herzegovina, the Bangladesh Community Engagement Initiative, the Polygraph Examination Program in Nepal, the Maritime Policing Program in the Philippines, the Ukraine Police Patrol Program, and the Youth Summer School in Pakistan.
To take one example, the Islamabad Capital Territory Police (ICTP) Youth Summer School was developed as part of a larger program ICITAP launched in Pakistan in 2002 aimed to improve seven law enforcement entities across a number of key areas. The Youth Summer School in particular, served to increase trust between the community and the ICTP, and addresses three of ICITAP’s specific program efforts – “organizational development, implementation of modern training practices, and integration of internationally accepted police practices.” ICITAP provides personnel and financial support for the camp, which serves over 300 children annually at no cost to families.
To evaluate their efforts, ICITAP utilized qualitative surveys to collect feedback on the program, and parents, youth, and ICTP constables alike have expressed strongly positive experiences. The majority of youth and parents report “an improved impression of the ICTP after the camp experience … [and expressed interest] in joining or having their child join the ICTP.” Moreover, ICTP constables have reported changing their perspectives and approach toward policing since participating in the camp.
In promoting the sustainability of this program, ICITAP hopes to establish camps in other areas in Pakistan and emphasizes that the camps should be replicated in locations where sponsoring agencies can provide resources on a long-term basis, and state that achievement of similar outcomes will require these areas to be characterized by high distrust between youth and police. To strengthen the program, ICITAP hopes to collect greater qualitative and quantitative data from families as well as police officers in order to test specific hypotheses regarding “changes in police academy enrollment among children who have attended the camp or … use of force by participating police officers.”
Monitoring and Evaluation
Monitoring and evaluation of ICITAP’s programs are regulated by the Government Performance and Results Act (GPRA), as well as the Foreign Aid Transparency and Accountability Act of 2016 (FATAA). These laws represent efforts to increase accountability and provide decision-makers with adequate information on the results of federal programs.
ICITAP currently utilizes a Training Events Log, which tracks all ICITAP training programs globally. This system is used to gather data on “course titles, number of training iterations provided, number of host-country nationals trained, and a separate listing of female students who attended the training.” To increase the quality and effectiveness of monitoring and evaluation, ICITAP has developed definitions for several relevant terms and is currently developing model program designs for its various core competency areas. ICITAP believes these developments will improve organizational cohesion, which they also hope will lead to increased clarity and consistency in their funding agreements.
The report provides examples of ICITAP’s recent monitoring and evaluation approaches in various contexts. In the case of the training program implemented in Pakistan, pre- and post-tests were utilized to measure student learning, and both quantitative and qualitative survey items were used to summarize student satisfaction. To measure how the training has affected the trainees’ performance, the report discusses the potential for self-report surveys, as well as surveys of the trainees’ commanding officers. The number of trained officers in various areas covered in the program courses is also suggested as an evaluation outcome measure.
The report notes the use of surveys as a critical tool for program monitoring and evaluation and argues that this method, when used properly, can provide far more useful information than statistics alone. In promoting the best use of surveys as an evaluation method, the report emphasizes several points. First, the survey should have a clear and explicit purpose to help generate realistic recommendations. Further, a reasonable sampling method and sample size should be utilized to ensure the results are generalizable, and the survey design should be appropriate for the demographic characteristics of respondents to help ensure a high response rate. Finally, cross-validation with additional data sources should be used whenever possible to ensure valid results, and the limitations of the surveys should be acknowledged and addressed as appropriate.
NOTE: This summary is produced by the Rule of Law Collaborative, not by the original author(s).