C-02

The Public Spending Code:

C. Implementation and Post-Implementation

Periodic Evaluation/Post-Project Review

C-02

Document Update Log

Document Summary: All expenditure is subject to ongoing monitoring using appropriate performance indicators. Ongoing analysis of performance indicators should give management a good idea of whether an investment or intervention is yielding the expected outputs and outcomes. A subset of expenditure in any one year will be subject to further in-depth evaluation. Evaluation/post-project review of some expenditure is mandatory i.e. capital projects > €20m whereas there is discretion on the selection of other projects/programmes/schemes that will be selected for evaluation. This document outlines why there is a need for more in-depth evaluations and what must be evaluated and also the importance of aligning evaluation timetables with the new ‘Whole of Year’ Budgetary process.

The importance of active management, regular reporting and monitoring and the use of performance indicators was outlined in Public Spending Code Document C-01 Management.  Active management allows a sponsoring agency to assess whether a capital project is on schedule and within budget. For capital grant schemes and current programmes a regular analysis of  performance indicators should give the sponsoring agency and sanctioning authority a good idea of whether an intervention is achieving its objectives or not.

In addition to the active management and regular analysis of performance indicators there is a need for periodic evaluations of areas of expenditure. This requirement is there because:

–          regular monitoring of performance indicators needs to be supplemented with a more in-depth study to assess efficiency and/or effectiveness

–          an independent review of efficiency, effectiveness and continued relevance is sometimes needed

–          the outcomes of the intervention will not occur for some time and a different approach to measuring effectiveness is required

–          the scale of the investment/intervention justifies an in-depth evaluation

For capital projects the benefits will not be seen until the project has been completed. The project has by then exited the active management stage. All large capital projects and a proportion of other capital projects have to be subjected to a post-project review to see if the predicted benefits of the project were realised. Post-project reviews should be undertaken once sufficient time has elapsed to allow the project to be properly evaluated with sufficient evidence of the flow of benefits/costs from it. There are two separate focuses of review – (i) project outturn and (ii) appraisal and management procedures. The second element can be done after project completion as it involves reviewing administrative and management procedures. The timing of the first element will depend on the nature of the project i.e. the period required to observe the expected benefits. This period should be no longer than one third of the timeframe used in the Appraisal.   The detailed appraisal provides the base against which the outturn review is made. The aim of a review of project outturn is to determine whether:

– the basis on which a project was undertaken proved correct;

– the expected benefits and outcomes materialised;

– the planned outcomes were the appropriate responses to actual public needs;

– the appraisal and management procedures adopted were satisfactory;

– conclusions can be drawn which are applicable to other projects; to the ongoing use of the asset; or to associated policies.

Post-project reviews for capital grant schemes and for current expenditure programmes may also be needed particularly where evaluations were not undertaken when the schemes were active or if the benefits would not be apparent for some time. Post-implementation reviews reveal if the type of intervention chosen is effective and efficient and informs future decision making.

The Value for Money and Policy Review process aims to subject some significant portion of an organization’s expenditure to an in-depth review every year. There are also more focused reviews that may not examine all of the evaluation questions posed by a VFMPR. (See Public Spending Code Document C-04 Reviewing and Assessing Expenditure Programmes).

Note:   The monitoring, management, evaluation or review of discrete areas of expenditure should incorporate the relevant administrative expenditure associated.

Evaluations and the Annual Estimates and Budgetary Timetable

Whether evaluations are undertaken as part of the VFMPR initiative, with a full set of terms of references or focused on a targeted subset of evaluation questions e.g. effectiveness or efficiency they should be completed within a reasonable period (6-9mths for full set of terms of reference and much less for more focused evaluations). They should be scheduled so that their findings are available for the forthcoming budgetary cycle.

From 2012 the budgetary process is moving to a ‘Whole of Year’ timetable. Oireachtas Committees will feed their views into the process starting in the spring of each year. It is expected that by the Autumn of each year Committees will be informed by the VFM reviews generated on an ongoing basis by Departments.

It is important therefore that Departments target the completion of their evaluations for the Autumn of each year at the latest so that the findings can inform opinions and decisions, in Departments, in the Committees and in the Department of Public Expenditure and Reform at the earliest opportunity. Failure to adhere to this schedule seriously undermines the value of the evaluation work. To give Departments their best chance of meeting this timetable significant new evaluations should begin in the Autumn/early Winter.

Mandatory Evaluation/Post-Project Review Requirements

  • Capital Grant Schemes with an annual value in excess of €30m and of five years or more duration to be subject to prior and mid-term evaluation at the beginning and mid-point of each five year cycle or as may be agreed with the Department of Public Expenditure & Reform.
  • All Capital Projects costing > €20m[1] are to be subject of a post-project review
  • At least 5% of other capital projects should be reviewed
  • The VFMPR process obliges Departments to carry out a minimum numbers of VFMs. This varies depending on the size of the Department. See Public Spending Code Document C-04 – Reviewing and Assessing Expenditure Programmes.

Additional Evaluation/Post-Project Review Requirements

Departments and agencies should not restrict themselves to the mandatory evaluation or post-project review requirements. From time to time it may be apparent that while not mandatory, an area of expenditure would benefit from a more in-depth review based on the picture the performance indicators paint or maybe because the performance indicators are not as informative as originally thought.

Communicating lessons learned

As with all parts of the Public Spending Code any significant lessons should be translated into changes in the Sponsoring Agency’s practices and communicated within the organization and to the sanctioning authority so that it can apply any general lessons learned to this Code or to supplementary information.

Responsibility for Evaluation/Review

It is the responsibility of the Sponsoring Agency to carry out the evaluations or post project reviews. Those conducting reviews and evaluations should not be the same people as conducted the appraisal or managed the implementation. VFM & Policy Reviews have specific requirements regarding Steering Committees and independent chairpersons.


[1] As the threshold for post-project review has been reduced from €30m to €20m, DPER will consider on a case by case basis whether projects costing between €20m and €30m and appraised when the previous threshold figure applied, will require a post-project review.