P3 Canada Fund Screening and Project Ranking Process

Report Summary

The objective of this audit was to provide the CEO, the PPP Canada Management Team and Board of Directors with assurance that the framework of governance, systems and processes and internal controls related to the P3 Canada Fund screening process were in place, suitably designed, consistently applied and effective.

Throughout the audit fieldwork, the audit team observed examples of how controls are properly designed and are being applied effectively. This has resulted in several examples of strong management practices which are listed below:

  • The authority, responsibility, and accountability for P3 Canada Fund screening and project ranking process are clear, well communicated and documents such as the Assessor’s Guide and the Other Levels of Government Committee Terms of Reference, which are aligned with the terms and conditions of the P3 Canada fund;
  • PPP Canada conducts a detailed qualitative review on the completeness of all P3 Canada Fund applications prior to evaluation and scoring and a detailed quantitative scoring assessment;
  • The evaluation and scoring groups assigned to review and assess P3 Canada Fund applications are made up of skilled and experienced staff from the Project Development and Investment business units;
  • Training was provided to all employees that were selected to assess project applications in order to ensure consistency and objectivity in the assessments. The training session included a review of the Assessor’s Guide and participants noted they were encouraged to ask questions during the training session; and,
  • The Project information System is used to help ensure quality and consistency in the screening and project ranking process and to simplify the screening and project ranking process as compared to previous rounds.

Overall, the audit found that PPP Canada has an adequate framework of governance, systems and processes and internal controls related to the P3 Canada Fund Screening & Project Ranking Process; however, opportunities for improvement were noted to strengthen the Round Five P3 Canada Fund Screening & Project Ranking process.

Recommendations

Management Responses

We recommend that management develop a practice to formally evidence participation and agreement with the final Evaluation Team assessment for audit trail purposes.

PPP Canada agrees with the audit recommendation that formal practices to evidence participation and agreement with final scoring group assessments should be developed.

The evaluation and scoring groups assigned to review and assess P3 Canada Fund applications are made up of skilled and experienced staff from the Project Development and Investment business units. Group members have multiple years’ experience evaluating and supporting P3 projects.

PPP Canada will develop a formal participation list for the evaluation and scoring groups responsible for project screening and ranking. Once the groups have reached consensus on project scores, all participants will be required to sign a routing slip as evidence of their agreement with the final scores. Final consensus documentation will be saved on P3C Docs in the respective folder for audit trail purposes.

We recommend that management develop a practice to evidence approval of OLG Committee decisions and include additional rationale for key OLG decisions regarding project ranking “cut-offs” and strategic project considerations.

PPP Canada agrees with the audit recommendation. PPP Canada has made significant progress in developing formal, systematic processes and internal controls to manage and support internal governance arrangements.

Moving forward, the evaluation and scoring team will prepare documentation surrounding its rationale and decision associated with project ranking cut offs. Furthermore, the Other Levels of Government Committee will continue to document its rationale and decision associated and the selection of strategic projects. The rationale for both the cut offs and strategic projects will be presented to the OLG Committee for consideration and approval. Supporting documentation will be saved on P3C Docs in the respective folder for audit trail purposes.

We recommend that management include a review of the application submission date as part of the qualitative assessment phase prior to assessing application merit and all applications received after the application deadline should be automatically disqualified from the screening process.

PPP Canada agrees with the audit recommendation.

PPP Canada conducts a detailed qualitative review on the completeness of all P3 Canada Fund applications prior to evaluation and scoring. Moving forward, all applications, including those from regional contacts, interlocutors and representatives from other levels of government, will be required to submit their files to a dedicated email address (PPPadministration@p3canada.ca) on the date and time prescribed in the application guide. Applications received after the prescribed deadline will be automatically disqualified from the screening process.

Project Development will institute a formal checklist procedure to ensure that mandatory criteria associated with the screening process are met.

It is recommended that management segregate duties associated with the Project Information and introduce a formal change management process to manage all changes, updates, or modifications to the Project Information System.

PPP Canada agrees with the audit recommendation. Over the 2012/13 planning period, PPP Canada led the implementation of an Electronic Records Management System (EDRMS). The system is designed to enhance PPP Canada’s capability to manage documents and records and strengthen information security. By implementing the EDRMS, PPP Canada has the ability to manage access rights controls and keep a clear audit trail on any modifications and deletions to document versions and their associated profiles. Moving forward, PPP Canada will manage the P3 Canada Fund Project Information System (PIS) within the EDRMS. Project Development will grant a select few resources with editing access rights to the PIS. Any changes made to the PIS will be managed by these resources and evidenced by an audit trail within the EDRMS.

It is recommended that management conduct a review of the scoring approach and methodology for future rounds to ensure clearly defined evaluation metrics are used which are consistent with the documented criteria and scoring grid.

PPP Canada agrees with the audit recommendation. PPP Canada continuously monitors and reviews the processes, procedures and approach used to execute each round of the P3 Canada Fund. Lessons learned and experienced gained from previous rounds are used to inform the development of processes and procedures for future rounds.

Work is currently underway to update and strengthen the P3 Canada Fund evaluation/ assessors guide to enhance consistency with the criteria and scoring grid. PPP Canada will continue to provide staff with application assessment training.

Supporting documentation, final project scores and their respective ranking will be updated on EDRMS in the respective folder for audit trail purposes.