128
Trial Monitoring: A Reference Manual for Practitioners
Finally, especially for long-term programmes, it can be useful to arrange for periodic impact-assessment studies by outside experts. Such evaluations can provide a new perspective and are more likely
to be free from unintended biases or self-congratulations than internal evaluations.
13.3 Practical considerations of monitoring programmes in assessing their impact
The following guidelines may be considered in defining and implementing a methodology to assess
project impact:
• Programme managers should select a clear methodology for performing the assessment and interpreting its findings. To this end, they should review writings on impact-assessment methods,
as well as studies regarding the efficiency and quality of justice systems. For OSCE monitoring
programmes, the experience of the OSCE and the CoE are particularly relevant.193
• It is useful for managers to set out objectives to be achieved in the short, medium and long term.
This can help to make realistic assessments of progress.
• Programmes should make assessments of their impact in all areas at regular intervals, at least
once per year. Yearly and mid-year budget reviews present good opportunities in this regard.
• The method of impact assessment chosen should be manageable within the resources and capacity of the programme.
• Programme managers should encourage discussion and brainstorming on assessment among
monitoring programme staff members, as well as seeking advice elsewhere in the organization
from colleagues who are responsible for project approval. If feasible, they may consult with outside actors, such as those who work on compiling statistical data on the effectiveness of the host
country’s justice system. Judicial and prosecutorial centres can be sources of information, as can
agencies specialized in system evaluation. Statistics and other indicators gathered from the authorities themselves or from other organizations can also be used.
• Tracking the extent to which programme recommendations have been implemented by authorities assists in providing an informed view about whether implementation has been successful.
• Although some programmes do not have the capacity to gather comprehensive statistics on all
issues relevant to their activities, managers can, nonetheless, review whether the authorities have
taken positive steps to implement recommendations in general terms.
• Programmes should not miss “the forest for the trees” in relation to judicial reforms, and should
prioritize their activities accordingly. At many points during the life of a systemic programme,
managers will need to make decisions about whether to continue certain activities, pay greater attention to specific issues, or launch activities in new fields. Managers should be resolute if
a change in focus or methods is appropriate. Resistance to change from within an organization
can often arise.
• Finally, it is necessary for programmes to be alert to any unintended effects of or impact from
their activities, and to take corrective action if necessary. For instance, if governmental actors
use monitoring reports as an excuse to retaliate against judges or defence attorneys whose practices may be reported, this would require a careful assessment by and reaction from programme
managers.
193 Also of interest is the website of the European Commission for the Efficiency of Justice, at <http://www.coe.int/t/dghl/
cooperation/cepej/default_en.asp>.
Select target paragraph3
Connect to a paragraph
Connect to an entity
Disable highlights
Add to table of contents