Audit of Canada First Research Excellence Fund

The Honourable François-Philippe Champagne, P.C., M.P.
Minister of Innovation, Science and Industry

© Her Majesty the Queen in Right of Canada,
represented by the Minister of Industry, 2021

Cat. No. CR22-121/2021E-PDF
ISSN 978-0-660-40302-1

PDF document (213 KB)

Table of contents

Executive summary

Background

The Canada First Research Excellence Fund (CFREF) (the Program) is a tri-Agency initiative of the Natural Sciences and Engineering Research Council (NSERC), the Social Sciences and Humanities Research Council (SSHRC) and the Canadian Institutes of Health Research (CIHR), collectively referred to as the Agencies.

CFREF invests approximately $200 million per year, helping Canadian post-secondary institutions (qualified universities, colleges and polytechnics) compete with the best in the world for talent and partnership opportunities, to make breakthrough discoveries, and to excel globally in research areas that will create long-term economic advantages for Canada. CFREF aims to support the establishment of institutional visions and long-term plans to achieve far-reaching, ambitious objectives that will lead to international recognition.

The Program is administered on behalf of the Agencies by the Tri-agency Institutional Programs Secretariat (TIPS) that is housed within SSHRC.

The Program had no prescribed size for awards in the first two (2) funding competitions. However, applicants were advised that each award was expected to support a broad, ambitious, institutional strategy that focused on the areas in which the institution could realistically achieve global leadership. As such, the Program contemplated research proposals that included funding from any number and combination of the three Agencies. Program funding was awarded for a period of up to seven (7) years, requiring long-term administration of the fund.

The two (2) competitions to date funded five (5) institutions in the first competition, with combined funding of $349 million, and 13 institutions in the second competition, with combined funding of $900 million.

The planned mid-term evaluation of the Program was being undertaken in 2019-20, the results of which were to be reported in advance of the third competition, scheduled to take place in 2021-22.

Why It Is Important

The initiation processes for TIPS programs have not been audited and other similar program audits have identified program management weaknesses. It is important to assess whether TIPS program initiation demonstrates similar control weaknesses.

This audit is included in the SSHRC 2018-21 Risk-Based Audit Plan (RBAP) that was approved in principle at the June 21, 2018 meeting of the Independent Audit Committee.

Audit Objective and Scope

The objective of this audit engagement is to provide assurance to the president of SSRHC that an adequate framework is in place to support the launch and ongoing management of the Program. The scope of the audit includes management processes and controls since the Program was announced in 2014-15. The audit scope excludes direct liaison with institutions that are contemplated under the Program.

Summary of Key Findings

CFREF was designed as a permanent grant program to administer funding on behalf of the Agencies, reporting administratively to SSHRC. The CFREF Program management (Program management) has administrative responsibilities throughout the life cycle of the granting process (8A’s model for grants ― Arrange, Advertise, Apply, Assess, Award, Admin, Audit, Acquit).

The audit found that the Program was well defined during its creation, given the complexities of a new, large value, tri-Agency grant program, including consideration of the needs of the three granting Agencies and numerous stakeholders.

The audit has identified opportunities to improve:

  • The administration of the Program risks and performance measurement to comply with the requirements of theTreasury Board Secretariat’s Policy on Transfer Payments as it relates to risk management, performance measurement strategies and service standards;
  • Administration of funding recipient requirements, including management of Program eligibility and compliance of Program funding recipients with the requirements of their respective Program funding agreement; and
  • Management and security of information to ensure information is accessible, understandable and usable in the management of the Program, and the accountability and security requirements of NSERC-SSHRC information management directives and standards are met.

Overall Conclusion

The short timeframe for the launch of the Program, combined with limited human and business resources, had an adverse effect on the administration of the Program.

The audit identified areas of weakness in the administration of the Program and in the design and effectiveness of management controls for management’s consideration. These processes and controls ensure Program management fulfills its responsibilities for administering the grant competition process, including a framework to manage risk and performance measurement, administration of funding recipient requirements and management of Program information. As part of this continued improvement, Program management should leverage TIPS and other Divisions within the Agencies to meet administrative responsibilities. These audit findings represent an opportunity for management to reflect and consider improvements going forward.

1. Background

CFREF is a tri-Agency initiative of NSERC, SSHRC and CIHR, collectively referred to as the Agencies. The CFREF program (the Program) was designed as a permanent grant program to administer funding on behalf of the Agencies, reporting administratively to SSHRC. The Program has administrative responsibilities for managing business processes throughout the life cycle of the granting process (8A’s model for grants ― Arrange, Advertise, Apply, Assess, Award, Admin, Audit, Acquit). SSHRC is assigned responsibility for evaluation and audit, and the Program is governed by the TIPS Steering Committee. As a program administered by SSHRC, the CFREF Program receives service support from other Divisions of NSERC and SSHRC, including corporate services such as finance, human resources and information technology.

The Program invests approximately $200 million per year to support Canada’s post-secondary institutions in their efforts to become global research leaders. CFREF helps Canadian post-secondary institutions (qualified universities, colleges and polytechnics) compete with the best in the world for talent and partnership opportunities, to make breakthrough discoveries and to excel globally in research areas that will create long-term economic advantages for Canada. CFREF aims to support the establishment of institutional visions and long-term plans to achieve far-reaching, ambitious objectives that will lead to international recognition.

There have been two (2) competitions for funding to date. In each competition, Program applicants were expected to submit proposals for funding that would support a broad, ambitious, institutional strategy that focused on the areas in which the institution could realistically achieve global leadership. The Program allowed for proposals that were cross-disciplinary, which included proposals for funding from any number and combination of the three Agencies. Program funding was awarded for a period of up to seven (7) years, requiring Program management to consider the long-term administration of the fund.

In the first competition in 2015, 36 institutions submitted applications, of which five (5) were selected for combined funding of $349,243,000. In the second competition in 2016, 29 institutions submitted applications, of which 13 were selected for combined funding of $900,000,000.

The planned mid-term evaluation of the Program is being undertaken in 2019-20, the results of which are to be reported before the third competition, scheduled to take place in 2021-22.

This audit is included in the SSHRC 2018-21 RBAP that was approved in principle at the June 21, 2018 meeting of the Independent Audit Committee.

2. Audit objective and scope

The objective of this audit engagement is to provide assurance to the president of SSRHC that an adequate framework is in place to support the launch and ongoing management of the Program. Of particular interest to this audit were Program management’s responsibilities for:

  1. Administering the Program’s risk management and performance measurement at both the grant recipient level and the overall Program level, in a manner that addresses the reporting requirements of each Agency.
  2. Administering the grant competition process by identifying institutions eligible for the Program and monitoring funding agreement requirements.
  3. Managing business information needs to ensure efficient and effective administration of the Program, and compliance with information management and security requirements.

The scope of the audit included management processes and controls since the Program was announced in 2014-15, including:

  • The processes and controls to award and manage the life cycle of funding competitions in 2015-16 and 2016-17;
  • The direct accountabilities and responsibilities of Program management and staff;
  • Program management’s roles and responsibilities for activities performed by other Divisions of NSERC and SSHRC on behalf of the Program; and
  • Documents sent to and received from institutions that applied for and received funding from the Program.

The audit scope excluded direct liaison with institutions that are contemplated under the Program.

3. Audit methodology

The audit was conducted internally by the Corporate Internal Audit Division. The audit team used the following methodology in the conduct of audit work.

  • Review of documentation and files of various sources of information from the Program and TIPS, including Program literature, policies, guidelines, the CFREF website, documents used to manage the life cycle of the Program, etc. This included consideration of process and control documents used to manage the Program, including individual Program institutional applicant and funding recipient files.
  • Questionnaires and interviews with management and staff directly responsible for the Program and TIPS program activities, and management of Divisions of NSERC and SSHRC that had roles and responsibilities for activities performed on behalf of the Program.
  • Testing the effectiveness of the controls over:
    • the inclusion of performance metrics in funding agreements and the corresponding timely reporting of performance by funding recipients;
    • the identification and timely reporting of deferred and lapsed funding.
    • the initial and ongoing assessment of institutional eligibility for the Program; and
    • the compliance of funding recipients with their respective funding agreement.

This audit conforms with the Institute of Internal Auditors’ International Professional Practices Framework, in accordance with the Government of Canada’s Policy on Internal Audit, as supported by the results of the quality assurance and improvement program. These standards require that sufficient and appropriate audit procedures be conducted and that evidence be gathered to provide a high level of assurance on the findings contained in this report. The audit conclusions are based on the audit findings against audit criteria included in Appendix I.

Peter Finnigan, Chief Audit Executive
Corporate Internal Audit Division, NSERC and SSHRC

4. Audit findings

The Program successfully delivered grants from two competitions in 2015-16 and 2016-17. Many of the audit findings reflect the context of managing a program that was both being developed and launched within a short timeframe, followed closely by two funding competitions, all within three (3) years.

The audit noted the inherent challenges of launching a Program within a short timeframe, with limited human and business resources. The Program also experienced turnover of key staff after its launch.

Management noted that the Program was assigned experienced staff by TIPS, and the Program made use of existing business tools and language used in program literature from other programs to mitigate risks associated with launching a new Program and running the competitions.

The audit found that Program management had limited time to prioritize, document and effectively conduct certain key activities. The audit found that improvements were necessary in the areas of risk management and performance measurement, administration of funding recipients and management of information.

4.1 Risk Management and Performance Measurement Frameworks

During the creation of the Program, management established its administrative responsibilities, recognizing the need to identify, manage and monitor the risks and performance of Program funding recipients and the overall Program.

A framework for managing Program risks and performance measurement was proposed as a strategy to mitigate risks identified during the creation of the Program. Program service standards were considered in the overall effective management of funding recipient performance, Program objectives and the objectives of each Agency. Risk management, performance measurement and service standards are also requirements of the Policy on Transfer Payments.

4.1.1 There was no formal risk management framework in place to identify, manage and monitor existing and emerging risks to the Program.

Risk Management

The audit expected to find a risk management framework, consistent with the strategy proposed during the creation of the Program. The Policy on Transfer Payments, section 6.5.2, indicates that the deputy head is accountable for risk management by “Ensuring that the administrative requirements on recipients are proportionate to the risk level. In particular, that monitoring, reporting and auditing reflect the risks specific to the program, the value of funding in relation to administrative costs, and the risk profile of the recipient.”Footnote 1

The 2012 Fall Report of the Auditor General of Canada–Grant and Contribution Program Reforms, chapter 2.34, emphasizes the relationship between reporting and risks, indicating that “According to the Policy on Transfer Payments, all new and continued grant and contribution programs approved after 31 March 2010 had to establish and apply a risk rating for each agreement and ensure that reporting requirements reflected the risk assessment.”

The audit found that the Program had some documentation related to the risk mitigation strategies that were proposed when the Program was created. Some Program risks were also included in the Program literature. According to the Program literature on the CFREF website, the criteria for selecting proposals included an assessment of the quality of the implementation and risk management plans included in each funding application.

The audit found that Program management did not effectively administer the risks of the Program consistent with the level of risk and the need for a risk management framework that was identified during the creation of the Program.

CFREF management did not have a formal plan to manage Program risks, including a methodology to identify, rate and manage risks throughout the Program life cycle.

It was not evident that Program management effectively addressed the requirements of the Policy on Transfer Payments, including the risk management requirements under section 6.5.7, for ensuring monitoring, reporting and auditing reflected the risks specific to the Program, the value of funding in relation to administrative costs, and the risk profile of the recipients.

4.1.2 There was no framework for managing the Program performance measurement.

Performance

The audit expected to find a framework for managing performance, consistent with the strategy proposed during the creation of the Program. The Policy on Transfer Payments, section 6.5.7, indicates that the deputy head is accountable for performance measurement by “Ensuring that a performance measurement strategy is established at the time of program design, and that it is maintained and updated throughout its life cycle, to effectively support an evaluation or review of relevance and effectiveness of each transfer payment program.”Footnote 2

According to the Program literature and guidance on the CFREF website, applicants were required to submit a performance measurement plan with their respective proposal, which would be the basis for ongoing reporting, progress monitoring and evaluating results during the mid-term evaluation.

The Program had some management tools to assess performance results and had defined the performance and financial monitoring activities. CFREF management initiated the development of a logic model for the performance measurement of Program-funded research. A mid-term evaluation of funded research was included in the mitigation strategy identified during the creation of the Program; the mid-term evaluation was in the conduct phase during this audit.

The Program did not have a complete documented plan to manage Program performance measurement and reporting throughout the Program life cycle nor guidance for staff to administer the performance assessment of Program recipients. The audit found limited evidence of performance assessments, and the documentation was not complete or consistent.

It was not evident that the Program had processes or controls to ensure performance measurement plans were identified in each funding agreement during the funding competitions and to ensure funding recipients reported in a timely manner, consistent with their respective funding agreement.

Performance Reporting

The Program was designed to provide applicants with an opportunity to access funding from any one of NSERC, SSHRC and CIHR including a combination of two or all three Agencies for a single funding award. The audit expected to find a framework for performance reporting necessary to meet the reporting requirements of each of the Agencies.

Funding consisted of awards from one, two and all three Agencies as part of a single award; however, Program management did not consider the performance reporting requirements of each Agency when it established the reporting requirements of the funding recipient. The Program did not consider how the performance results would be attributed to each Agency when funding was awarded by more than one Agency for a single proposal. Further, the Program did not consider how it would provide assurance and performance reporting to each Agency regarding its individual accountability and reporting requirements under the Policy on Transfer Payments.

4.1.3 The Program did not establish and manage service standards.

Service Standards

The audit expected to find that service standards had been developed and used to manage the Program. The Policy on Transfer Payments, section 6.5.9, indicates that the deputy head is accountable for “Establishing reasonable and practical departmental service standards for transfer payment programs.”Footnote 3

CFREF management did not establish or report against Program service standards throughout the life cycle of the granting process (8A’s model for grants ― Arrange, Advertise, Apply, Assess, Award, Admin, Audit, Acquit). Further, the audit found that the Program did not have service agreements with other Divisions within the Agencies for establishing reasonable and practical service standards.

There are opportunities to improve the effective administration of the risks and performance measurement of the Program. The absence of defined risk management and performance management frameworks may result in unclear roles, responsibilities and accountabilities for managing and monitoring risks to an acceptably low level. CFREF management may not be able to effectively analyze and assess the demonstrated need for funds.

The Program may not be compliant with the requirements of the Policy on Transfer Payments as it relates to risk management, performance measurement strategies and service standards.

Recommendation 1

It is recommended that the associate vice-president of TIPS:

  • Establish a framework to effectively manage the administration of Program risks, performance measurement and service standards. The framework should be designed to ensure that it addresses the accountability and reporting requirements under the Policy on Transfer Payments for each Agency; and
  • Establish formal service agreements with other Divisions within the Agencies to effectively leverage existing Program services.

Management Response and Action Plan

Agree. Program management plans to complete work by Fall 2022 to:

  • Create and implement a framework to manage risks, performance measurement and service standards. The framework will comply with requirements under the Policy on Transfer Payments for each Agency;
  • Assess options that better support the administration of the program and continue to engage and leverage existing key groups at SSHRC-NSERC, such as the Finance, Information Technology and Information Systems and Corporate Strategy and Performance Divisions. Options could include, but are not limited to, the development of formal service agreements.

4.2 Administration of Funding Recipients

The Program was assigned the responsibility for administering the grant competition process, including the financial aspects of the Program, performance measurement, monitoring and reporting.

Eligibility

Each Agency is responsible for determining whether institutions meet the general eligibility and ongoing compliance requirements to receive funding from their respective Agency. Eligibility for a specific fund is performed at a program level; eligibility to receive funding from an Agency does not automatically qualify an institution to receive funding from an individual fund. The Program established funding eligibility criteria that were specific to the Program and supplemental to the funding eligibility requirements of each Agency. The Program also included specific Program compliance requirements in each of the funding recipient agreements.

In so doing, the Program assumed responsibility for monitoring recipients to ensure they were eligible for Program funding and compliant with the specific requirements of their respective funding agreement. Eligibility for funding by the Agencies did not assure eligibility for Program funding. An institution could be generally eligible to receive grants from an Agency; however, the institution would still have to meet the eligibility requirements of the Program to receive funding.

Based on the significance of institutional eligibility to the Program, it was expected that management had defined requirements for institutional eligibility for Program funding, guidance on how institutional eligibility would be determined in advance of awarding funding, a determination of eligibility of Program recipients on an ongoing basis and documentation of a business process (e.g. assessment, review, approval, reporting, archiving, etc.).

CFREF management recognized the significance of defining eligibility requirements for institutions to qualify for Program funding throughout the life cycle of the grant. The audit found that:

  • Management identified risks associated with eligibility in the initial implementation phase of the Program;
  • Eligibility requirements were defined and included in Program literature, made public on the CFREF website; and
  • Eligibility requirements were embedded in the guidance for funding applicants, which made Program funding contingent on initial and continued eligibility.

4.2.1 The Program did not account for and manage differences between the institutional eligibility requirements of the Program and the institutional eligibility requirements for grant funding from each Agency.

The audit found that the assessment of institutional eligibility by each Agency was a task performed by Divisions independent of the Program. The ongoing assessment of institutions to administer funds was performed by each Agency and did not consider the specific eligibility requirements of individual programs, including the eligibility requirements of Program funding applicants.

In addition, it was not evident that the Program roles and responsibilities for assessing and monitoring the eligibility of institutions for the Program and the shared roles and responsibilities with other supporting Divisions at each Agency to assess and monitor institutional eligibility were well understood by Program management. Further, Program management did not account for and manage differences between the eligibility of institutions for grant funding from the Program and the eligibility of institutions for grant funding from each Agency.

The audit tested the Program management controls that ensured the initial and ongoing eligibility of institutions for Program funding. The test concluded that the controls were not effectively designed to prevent or detect ineligibility of funding applicants and recipients. Program management provided evidence of their assessment of the eligibility of institutions at the time they applied for CFREF funding; however, the evidence did not demonstrate that there were adequate processes or controls to ensure the initial and ongoing eligibility of institutions for Program funding.

4.2.2 Program management did not assess, monitor and report compliance of institutions with the requirements of their respective Program funding agreement.

Compliance Monitoring

Compliance requirements were developed when the Program was established. These requirements were documented in a variety of sources, including Program literature, guidance, operational procedures applicable to the administration of the Program, SSHRC policies and directives, and in broader policies and directives applicable across the Agencies.

It was expected that Program management had processes and controls to ensure funding recipients were compliant with the funding agreement requirements of the respective Agencies. It was expected that Program management had formally documented roles, responsibilities and management tools for monitoring, managing and reporting of postsecondary institutions’ compliance with their respective funding agreement. It was expected that Program management had processes and controls to effectively mitigate the risk that the Program exceeded deferred funding thresholds and the risk that research funding lapsed. It was also expected that the Program had adequate controls to ensure accurate reporting of grants, based on Proactive Disclosure requirements.

The Program requirements were included in each funding recipient agreement. Program funding was contingent on compliance of each funding recipient with the requirements of their respective funding agreement.

The audit found that the funding agreements included clauses with specific requirements for the funding recipient to comply with institutional agreements, TIPS program guidance and policies, and policies, directives, guidance, etc. that were external to the funding agreement, including all the policies of each Agency.

These external requirements had a compounding effect on the funding recipient. For example, the individual funding recipient agreements required compliance with the CFREF Administration Guide. The CFREF Administration Guide required compliance with the Agreement on the Administration of Agency Grants and Awards by Research Institutions section 4.9, Agency Policies, that indicates “The Institution shall comply with their responsibilities in accordance with all relevant Agency policies, as amended or introduced within the duration of the Agreement”Footnote 4 for Agency policies.

The audit found that the Program’s administrative roles and responsibilities to assess, monitor and report compliance of institutions with the requirements of their respective funding agreement were not well understood by Program management. As such, Program controls were not designed to ensure funding recipient compliance. It was not evident that the Program:

  • Had guidance or an overall process to manage compliance requirements;
  • Differentiated and prioritized funding agreement requirements that were of greater risk;
  • Managed compliance with the specific requirements of the Program; and
  • Considered how compliance with externally referenced requirements would be ensured.

The audit found that other supporting Divisions of each Agency were not responsible for assessing the compliance requirements of individual programs, such as the CFREF program.

The audit tested the Program management controls and concluded that the controls were not effectively designed to ensure funding recipients were compliant with the funding agreement requirements of each respective funding Agency. Improvements are required to better:

  • Perform an initial assessment of compliance of funding recipients at the time Program funding is awarded;
  • Maintain a listing or other definitive source of funding recipient compliance requirements; and
  • Assess, monitor and report the compliance of Program funding recipients with their respective funding agreement.

The audit tested the Program management controls to ensure that funding recipients that had exceeded the deferred funding threshold or whose funding had lapsed were identified and reported in a timely manner. The test concluded that the Program management controls were not effectively designed to prevent or detect lapses or deferrals. Improvements are required to better:

  • Monitor and report deferred funding thresholds and ensure funding has not lapsed;
  • Manage and report Proactive Disclosure; and
  • Collaborate with other Divisions within NSERC-SSHRC to effectively manage lapsed and deferred funding and Proactive Disclosure.

It was not evident that there was a clear division of roles and responsibilities between the Program and other Divisions of each Agency to assess, monitor and report funding eligibility and compliance of institutions with the requirements of their respective funding agreements.

The gap in Program controls may result in funding being awarded to institutions that are not compliant with their respective funding agreement and funding being awarded to institutions that may not be eligible for Program funding. The gap in controls may result in deferred Program funding that exceeds thresholds and is not detected, and in undetected lapsed Program funding. Further, the gap in controls may result in inaccurate Proactive Disclosure reporting.

Recommendation 2

It is recommended that the associate vice-president of TIPS:

  • Define the Program’s formal roles and responsibilities for assessing, monitoring and reporting the institutional eligibility and Program funding recipient requirements, including the shared roles and responsibilities with other Divisions of each Agency;
  • Ensure the Program has adequate and appropriate management processes and controls to assess and approve eligibility of institutions for Program funding;
  • Ensure Program funding recipients are compliant with the requirements of the Program, using a risk-based approach; and
  • Develop guidance on how and when institutional eligibility and Program funding recipient compliance is assessed and monitored throughout the funding life cycle, ensuring there are adequate business processes (e.g. assessment, review, approval, reporting, archiving, etc.), in collaboration with each Agency.

Management Response and Action Plan

Partially agree. Program management remains confident in the eligibility of current grant recipients, both upon approval of the grant and on an on-going basis. Nevertheless, Program management plans to complete work by Summer 2022 to:

  • Establish and document formal roles, procedures and timelines for verifying eligibility of current grants at a program and agency level;
  • Create a process that determines the eligibility of institutions to seek funding from the program. This will support the next CFREF competition.

4.3 Information Management

The Program assumed administrative responsibilities for information management as part of discharging its responsibilities for the financial aspects of the Program, performance measurement, monitoring and reporting.

NSERC-SSHRC issued directives and standards to address information management issues, such as management accountability, management of government information, access to information and protection of personal information, to satisfy the information needs of government. For example, the NSERC-SSHRC Directive on Records Management is intended to ensure records are managed in a manner that allows NSERC-SSHRC to meet legal, business and accountability requirements and remain accessible, understandable and usable for as long as they are required. The Directive on Information Security Management requires NSERC-SSHRC employees to classify, manage and secure sensitive information. Another example includes the Standard Information Structure that is intended to be used to structure information within systems such as Enterprise, and to support employees in managing programs and conducting their daily activities.

The Agencies, including TIPS and the Program, shared a long-term strategy for a Tri-agency Grants Management Solution (TGMS). Once implemented, TGMS is expected to mitigate risks associated with information technology, knowledge management, and information management, privacy and stewardship. TGMS has a discovery phase completion target date of March 31, 2021, with the implementation phase to follow.

It was expected that Program management would have a high-level strategy to manage information needs through the lifecycle of the funding Program (8A’s model for grants ― Arrange, Advertise, Apply, Assess, Award, Admin, Audit, Acquit).

4.3.1 The Program systems and processes did not adequately support the information management requirements of managers and staff in the discharge of their responsibilities.

The audit found that the Program did not undertake a formal information needs assessment. The Program did not develop an interim strategy to mitigate risks that TGMS is intended to address, including risks associated with information technology, knowledge management, and information management, privacy and stewardship.

The Program was managed using a combination of systems (including a SharePoint site and Enterprise) and documents (including spreadsheets, emails, word processing software, pdf and other documents). Formal information management guidance or processes were not clearly used in the creation and administration of the Program. This resulted in an unstructured approach to managing information needed to adequately support managers and staff in discharging their responsibilities.

CFREF management asserted that the Program made use of existing business tools and experienced staff, especially during the launch of the Program and funding competitions. CFREF management also noted that information management capabilities and the suite of systems being used were less than adequate, requiring a significant amount of manual processing (Enterprise, SharePoint external portal, CRM for financial data, password protection of documents).

It was not evident that Program management followed NSERC-SSHRC information management directives and standards, including the Directive on Records Management, to ensure information management met business and accountability requirements. The sensitivity of information was not apparent on the documents provided during the audit as required by the Directive on Information Security Management. The Program did not clearly address the information management requirements in the Standard Information Structure beyond the creation of the initial folder structure during the implementation of the Program.

It was not evident that Program management was certain that Program staff had appropriate system access and permissions to Program documents.

Without adequate information controls, information may not be accessible, understandable and usable in the management of the Program. Further, the management of Program information may not meet the accountability and security requirements of NSERC-SSHRC information management directives and standards.

Recommendation 3

It is recommended that the associate vice-president of TIPS:

  • Assess critical information needs consistent with the life cycle of the Program granting process (8A’s model for grants ― Arrange, Advertise, Apply, Assess, Award, Admin, Audit, Acquit);
  • Determine the Program’s information management roles and responsibilities, given the manual nature of information management for the Program, to ensure it is compliant with information management and security requirements; and
  • Develop mitigation strategies for information management as an interim solution before the launch of TGMS.

Management Response and Action Plan

Agree. Program management plans to complete work by September 2022 to:

  • Engage the Division, Information and Innovation Solutions (IIS) and other groups to determine if our existing systems, files and structures meet the current standards for information management and security requirements, and better define the roles and responsibilities of program staff;
  • Engage with IIS to develop a plan to onboard CFREF into Convergence for the next competition;
  • Leverage the Convergence portal as an interim solution with the goal of onboarding to TGMS once it is in place.

5. Overall conclusion

The audit identified several positive findings in areas linked to the administration of the Program. Most noteworthy were findings related to the establishment of the Program, which involved consideration of the complexities of a new, large-value grant program, including consideration of three granting Agencies and numerous stakeholders. The audit noted that the Program was well defined during its creation.

Management assigned experienced staff and made use of existing business tools and program literature to mitigate risks associated with launching a new program and running the competitions. The short timeframe for the launch of the Program, combined with limited human and business resources, had an adverse effect on the administration of the Program.

The audit identified areas of weakness in the administration of the Program and in the design and effectiveness of management controls for CFERF management’s consideration. These include the effectiveness of processes and controls to ensure Program management fulfills its responsibilities for administering the grant competition process, including risk management and performance measurement, administration of funding recipients and management of information. As part of this continued improvement, the Program should leverage TIPS and other Divisions within the Agencies to meet administrative responsibilities.

The Corporate Internal Audit Division would like to acknowledge and thank management and staff for their support throughout the conduct of this audit.

6. Audit team

Chief Audit Executive: Peter Finnigan
Internal Audit Principal: Mohamed Ayachi
Senior Internal Auditor: Dan Murphy
Senior Internal Auditor: Samir Harrabi
Senior Internal Auditor: Seymour Sambour

Appendix I – Audit lines of enquiry and criteria

The following areas of examination and the associated criteria were derived during the audit planning phase.

  1. Eligibility
    • Criteria 1.1: The Program management controls ensure the initial and ongoing eligibility of institutions for funding.
    • Criteria 1.2: The Program has clearly defined roles and responsibilities to identify and monitor the eligibility of institutions and the eligibility process is implemented in a timely manner.
  2. Risk Management and Performance
    • Criteria 2.1: The Program management controls ensure performance metrics are identified in each institution funding agreement and ensure funding recipient reporting is timely and consistent with the respective funding agreement.
    • Criteria 2.2: A framework for managing the Program risks exists, including roles, responsibilities and accountabilities for managing and monitoring risks, and is correlated with the organizations’ risk framework (TIPS, NSERC, SSHRC and CIHR).
    • Criteria 2.3 A framework for managing the Program performance exists (objectives, risks, performance metrics and assessment process) and is correlated with the Departmental Results for each of NSERC, SSHRC and CIHR.
    • Criteria 2.4: The Program has a framework for managing and monitoring Program service standards.
  3. Compliance
    • Criteria 3.1: The Program management controls ensure institution funding recipients are compliant with the funding agreement requirements of the respective funding Agency.
    • Criteria 3.2: The Program management controls ensure that any institution that has exceeded deferred funding thresholds or whose funding has lapsed is identified and reported in a timely manner.
    • Criteria 3.3: The Program has clearly defined roles and responsibilities for managers and staff to monitor compliance with funding requirements.
    • Criteria 3.4: The Program has clearly defined roles and responsibilities for managers and staff to monitor institutions for lapsed funding.
  4. Information Management
    • Criteria 4.1: The Program systems and processes adequately support the information management requirements of managers and staff in the discharge of their responsibilities.

Appendix II – Management response and action plans to audit reccomendations

Item Recommendation Management Response Target Date
1

It is recommended that the associate vice-president of TIPS:

  • Establish a framework to effectively manage the administration of Program risks, performance measurement and service standards. The framework should be designed to ensure that it addresses the accountability and reporting requirements under the Policy on Transfer Payments for each Agency; and
  • Establish formal service agreements with other Divisions within the Agencies to effectively leverage existing Program services.

Agree.

  • Create and implement a framework to manage risks, performance measurement and service standards. The framework will comply with requirements under the Policy on Transfer Payments for each Agency;
  • Program management will assess options that better support the administration of the program and continue to engage and leverage existing key groups at SSHRC-NSERC such as the Finance, Information Technology and Information Systems, and Corporate Strategy and Performance Divisions. Options could include, but are not limited to, the development of formal service agreements.
 Fall 2022
2

It is recommended that the associate vice-president of TIPS:

  • Define the Program’s formal roles and responsibilities for assessing, monitoring and reporting the institutional eligibility and Program funding recipient requirements, including the shared roles and responsibilities with other Divisions of each Agency;
  • Ensure the Program has adequate and appropriate management processes and controls to assess and approve eligibility of institutions for Program funding;
  • Ensure Program funding recipients are compliant with the requirements of the Program, using a risk-based approach; and
  • Develop guidance on how and when institutional eligibility and Program funding recipient compliance is assessed and monitored throughout the funding life cycle, ensuring there are adequate business processes (e.g. assessment, review, approval, reporting, archiving, etc.), in collaboration with each Agency.

Partially agree as Program management remains confident in the eligibility of current grant recipients, both upon approval of the grant and on an on-going basis. Nevertheless, Program management will:

  • Establish and document formal roles, procedures and timelines for verifying eligibility of current grants at a program and Agency level;
  • Create a process that determines the eligibility of institutions to seek funding from the program. This will support the next CFREF competition.
 Spring 2022
3

It is recommended that the associate vice-president of TIPS:

  • Assess critical information needs consistent with the life cycle of the Program granting process (8A’s model for grants ― Arrange, Advertise, Apply, Assess, Award, Admin, Audit, Acquit);
  • Determine the Program’s information management roles and responsibilities, given the manual nature of information management for the Program, to ensure it is compliant with information management and security requirements; and
  • Develop mitigation strategies for information management as an interim solution before the launch of TGMS.

Agree.

  • Engage the Division, Information and Innovation Solutions (IIS) and other groups to determine if our existing systems, files and structures meet the current standards for information management and security requirements, and better define the roles and responsibilities of program staff;
  • Engage with IIS to develop a plan to onboard CFREF onto Convergence for the next competition;
  • Leverage the Convergence portal as an interim solution with the goal of onboarding to TGMS once it is in place.
Sept. 2022

Footnotes

Footnote 1

Policy on Transfer Payments, section 6.5.7

Return to footnote 1 referrer

Footnote 2

Policy on Transfer Payments, section 6.5.2

Return to footnote 2 referrer

Footnote 3

Policy on Transfer Payments, section 6.5.9

Return to footnote 3 referrer

Footnote 4

Agreement on the Administration of Agency Grants and Awards by Research Institutions

Return to footnote 4 referrer