Internal Audit of Enterprise Performance Measurement Systems

Report Summary

The objectives of this review level assurance engagement were to determine if PPP Canada’s Enterprise Performance Measurement System (EPMS) is consistent with leading practices in the Federal government; and, determine if PPP Canada’s EPMS and supporting frameworks are suitably implemented and support achievement of the mandate.

Since its inception, PPP Canada has developed and refined its EPMS. PPP Canada recently updated its EPMS based on three areas: best practice expectations for EPM, review of other federal EPM approaches, and review of existing PPP Canada performance measures. Through analysis of documents, best practices and interviews, PPP Canada revised its EPMS to include an updated logic model and a refreshed performance measurement framework (PMF). PPP Canada’s EPMS identifies specific data sources for each logic model element and indicates in parentheses the department responsible for collecting the data. Frequency of data collection is identified.

PPP Canada presents and reports performance information using a range of systems. Reviews of PPP Canada’s Annual Reports for 2011-2012 and 2012-2013 find clear reporting of business line strategies, status updates against goals, and quantitative and qualitative descriptions of performance achievements. Reporting of procurement models by year, analysis of P3 marketplace, and indications of movement in key strategic sectors such as new P3 markets are reported. As well, descriptions of annual performance for all three business lines are reported. Indicators relevant to each business line are also reported.

Based upon the results of the review and pending implementation of opportunities for improvement the audit identified that PPP Canada’s EPMS is consistent with leading practices in the Federal Government and that supporting frameworks are identified and in place. Opportunities for improvement will further enhance the EPMS as they are implemented.

Recommendations

Management Responses

PPP Canada should reorganize its existing corporate profile to address all six profile elements, augmenting where necessary.

PPP Canada agrees with this recommendation. Despite the absence of a “program profile” for the P3 Canada Fund, all of the information required for the profile has already been developed and is included in corporate planning and reporting documentation, terms and conditions, and the formative evaluation. PPP Canada will compile the required information to develop a program profile for the P3 Canada Fund.

PPP Canada should develop a logic model narrative following TBS guidance.

PPP Canada agrees with this recommendation. PPP Canada will develop an accompanying narrative to the logic model. The narrative will outline how certain activities or actions are intended to produce results. This is sometimes referred to as the "theory of change" or "program theory." The narrative will explain the linkages between activities, outputs and outcomes by describing the underlying assumptions of the program, risks and external factors that influence whether or not the outcomes will be achieved.

Introduce baseline measures into PPP Canada’s PMF

PPP Canada agrees with this recommendation. Introducing baseline measures will enhance PPP Canada’s ability to tell the performance story over passage of time. PPP Canada will update the Performance Management Framework (PMF) to include baseline measures where appropriate.

Update PMF to include definitions and methodological descriptions of relevant performance indicators

PPP Canada agrees with this recommendation. PPP Canada will update the PMF to include data sources and methods required to support the PMF. 

As PPP gains experience tracking performance with the PMF, it should include performance to date against strategic objectives.

PPP Canada agrees with this recommendation. As part of the strategic planning and reporting cycle, PPP Canada integrates performance data into its corporate plan and annual report. These documents form the basis for business unit plans. PPP Canada will continue to continuously monitor, update and report on business results and targets as well as the economy and efficiency of their management.