Guidelines for Review Panel Members—Letter of Intent Stage

Background

The Canada First Research Excellence Fund (CFREF) is a tri-agency initiative of the Social Sciences and Humanities Research Council (SSHRC), the Natural Sciences and Engineering Research Council (NSERC) and the Canadian Institutes of Health Research (CIHR). It is administered by the Tri-agency Institutional Programs Secretariat, which is housed within SSHRC. The competition was announced in November 2021 to allocate up to $1.4 billion to support large-scale research programs focusing on a strategic thematic area related to the institution’s research excellence strengths and aligned with the Government of Canada’s Science, Technology and Innovation (ST&I) priorities for the CFREF and Canada Excellence Research Chairs programs (CERC).

CFREF aims to boost the strengths of Canadian postsecondary institutions so that they can achieve global success in research areas that create long-term social and economic advantages for Canada.

CFREF invests approximately $200 million per year through a highly competitive peer review process to support selected Canadian postsecondary institutions in turning their key strengths into world-leading capabilities.

CFREF objectives are to support the full range of research—from fundamental to applied—and to give institutions the ability to:

• pursue the best in the world for talent and partnership opportunities to enable breakthrough discoveries;
• seize emerging opportunities and strategically advance their greatest strengths on the global stage; and
• implement large-scale, transformational and forward-thinking institutional strategies.

Incremental and small-step evolution of research objectives will not meet the goals of this program.

There is no maximum or minimum limit for the grant value, and the number of grants depends on the amounts requested in successful applications. Each institution can submit only one Letter of Intent (LOI) / application as the lead institution. There is no limit to the number of applications on which an institution can be listed as a partner.

If you have any questions, do not hesitate to contact the program at cfref-apogee@chairs-chaires.gc.ca.

If you are experiencing technical problems (e.g., difficulty viewing or downloading electronic forms), contact: websupport@chairs-chaires.gc.ca.

Overview of the LOI evaluation process

Awards are made following a rigorous, competitive, international peer review process. LOIs undergo assessment by a multidisciplinary Review Panel before being sent for final approval to the Steering Committee.

• Review Panel—A multidisciplinary Review Panel will evaluate in detail the relative strengths and weaknesses of submitted LOIs, based on the selection criteria. The Review Panel provides recommendations regarding the most meritorious pool of LOI applicants to invite to submit a full proposal, and, ultimately, competitively award the funds available to proposals demonstrating the greatest potential to achieve world-leading research outcomes in areas that promise long-term economic advantages for Canada.
• Steering Committee—The Steering Committee is composed of the presidents of CIHR, NSERC, SSHRC and the Canada Foundation for Innovation (as an observer), as well as the deputy ministers of Innovation, Science and Economic Development Canada and Health Canada. The committee reviews the adjudication process to ensure that it was rigorous, objective, transparent and consistent with the program’s objectives. Based on the Review Panel’s recommendations, the Steering Committee provides final approval of the LOI applicants to invite to submit a full proposal.

Equity, diversity and inclusion

Equity, diversity and inclusion (EDI) is embedded as a foundational principle in CFREF’s objectives, expected outcomes, and application and reporting requirements. For the program to achieve its objectives and outcomes based on research excellence, the participation and contributions of students, trainees, personnel and researchers from underrepresented groups are required. Underrepresented groups include, but are not limited to, racialized minorities, Indigenous Peoples, persons with disabilities, women, and individuals from LGBTQ2+ communities.

Supporting early career researchers (ECRs) is a tri-agency priority as it enhances Canada’s position as a world leader in building talent and strengthening the research ecosystem. It is expected that CFREF initiatives will implement measures to specifically support ECRs.

EDI is essential to creating the excellent, innovative and impactful research necessary to seize opportunities and respond to global challenges.

The core personnel are expected to play key administrative and scientific leadership roles. It is expected that the approach taken by the institution to constitute its core team will reflect and be in line with its proposed EDI strategy to establish and maintain a diverse research team. The commitment made by institutions to excellence through diversity should be reflected in the composition of the core team, to convince assessors of the excellence of the team and its capacity to innovate and be impactful. Therefore, the core personnel’s composition should be assessed in relation to both Criterion 1. Scientific merit and demonstrated capacity to lead on an international scale, and Criterion 3. Quality of implementation plan.

All reviewers are asked to consistently guard against the possibility of unconscious bias influencing the decision-making process, whether this bias is based on a school of thought; fundamental versus applied research; certain subdisciplines; areas of research or approaches (including emerging ones); size or reputation of an institution; or the age, personal factors, sex or gender of the applicants. All reviewers are cautioned against judging an application based on these factors. Before review panel members are able to view any applications, they must first complete the required Unconscious Bias Training Module.

For applications involving Indigenous research, panel members should familiarize themselves with SSHRC’s Indigenous Research Statement of Principles and Guidelines for the Merit Review of Indigenous Research (specifically, the section describing “Key Concepts for the Merit Review of Indigenous Research”).

The panel will include EDI Champions.

Accessing the review material

You must complete your review using the Convergence Portal. The portal is supported only on the latest versions of Google Chrome, Microsoft Edge, Apple Safari and Mozilla Firefox. The portal may appear to function in other browsers, but technical problems can occur, such as information being improperly captured in the system without the user being aware. Use of an unsupported browser is strongly discouraged.

As a panel member, you will receive an email with detailed instructions asking you to log into (or create an account in) the Convergence Portal to complete your profile and accept the reviewer terms and conditions. After accepting the terms and conditions, you will have access to the reviewer dashboard in the Convergence Portal. From your dashboard, you must first declare any conflict of interest (COI), if applicable. The Conflicts of Interest tab will provide you a summary of the LOIs in order for you to make a determination of a COI.

Once the LOIs are assigned to panel members, you will have access to the Committee Assignments, where you will have access to a PDF copy of each applicant’s submission, including both their input into modules in the Convergence Portal and the supporting documents they prepared according to LOI instructions.

The PDF document is organized as follows:

Application details

• List of partner institutions
• List of partners
• List of core personnel
• Fields of research
• Keywords
• Summary of the implementation plan
• Summary of the scientific strategy
• Proposed budget
• Supporting document: Research program proposal (12 pages in English, 14.4 pages in French)
• Supporting document: Research program proposal references (no page limit)
• Supporting document: Budget table
• Supporting document: EDI plan (3 pages in English, 3.6 pages in French)
• Supporting document: Letters of support from partner institutions (2 pages per letter in English, 2.4 pages per letter in French)
• Supporting document: Core personnel biosketches (20 pages in English, 24 pages in French)
• Supporting document: Previously funded CFREF initiatives, if applicable (2 pages in English, 2.4 pages in French)
• Supporting document: Government of Canada’s science, technology and innovation priority research areas

LOIs submitted in French

LOIs submitted in French are assigned to panel members who have a reading comprehension of the French language.

Submitting your review (ratings and comments)

Each LOI will be assigned to three or more panel members, taking into account expertise, conflicts of interest, language and workload balance. You may be assigned to review LOIs outside of your area of scientific expertise (i.e., areas with which you are less comfortable). This is to help calibrate ratings across a spectrum of LOIs, as well as balance the workload across the multidisciplinary review panel.

After reading these guidelines and the LOI, follow the instructions provided in the Convergence Portal to complete and submit your ratings and comments for each of the three (3) evaluation criteria.

To assign ratings, please refer to the Definition of Ratings. In the comment boxes in the Convergence Portal, please provide objective feedback describing both the strengths and weaknesses of the LOI for each element of the criteria. Your written comments should be clear and concise, using objective and non-inflammatory language. Because your comments will be provided as feedback to the applicant, you should ensure that you provide sufficient detail. To ensure the confidentiality of the review process, please avoid providing information that identifies you.

When you are ready to submit your assessment in the Convergence Portal, review your ratings and comments to ensure they are complete. Click the “Submit Assessment” button to complete the assessment of each assigned LOI. The status of your submission will change to “Assessed.” Once all assessments are completed, select “CFREF-2022-1” from the Competition dropdown list, then select “Letter of Intent” from the Stage dropdown list. Finally, click the “Submit All Assessments” button. The status of your submission will change to “Submitted.” No changes can be made at that point.

We ask that your reviews be completed in the Convergence Portal by the date specified in your email invitation.

Review Panel meeting (teleconference)

A teleconference, moderated by the Multidisciplinary Review Panel chair, will be held to discuss individual LOI scores, especially where there is divergence among the assigned members (significant discrepancy/disagreement). The teleconference provides an additional forum to discuss the LOIs, share perspectives and consider various opinions. All panel members will be invited to participate in the discussion of the LOIs.

Ratings will be considered divergent if ratings provided by the assigned members do not accord―for example, a mix of Fully Satisfies (FS) or Fully Satisfies & Exceeds (FSE) together with Mostly Satisfies (MS) or Does Not Satisfy (DNS) for criteria 1 and 3 (see rating scale below), and a mix of "yes" and "no" for criterion 2.

To resolve divergent ratings, assigned members will not be required to modify their individual ratings, but the panel will be asked to reach a consensus rating for each criterion. In the interest of time, LOIs having received an individual rating of “no” on criterion 2 from all panel members will only be discussed to confirm the consensus for that rating. Applicants will receive both individual panel member ratings and consensus panel ratings.

Suggest external expert reviewers

We ask that you please suggest (if possible) up to five (5) expert reviewers (or more)―including international experts―with appropriate expertise to review the application in the event that the lead institution is invited to submit a full application (please provide their name, affiliation, email address).

Evaluation criteria and rating scale

Definition of ratings details the three (3) evaluation criteria and the rating scale you will use to score the LOIs assigned to you.

LOI instructions provide the instructions given to applicants, for your information.

Rating scale

After the panel reviews are completed, each rating will be converted to a numerical score to create a ranking of the LOIs for the Steering Committee’s consideration:

• Fully satisfies and exceeds (FSE)
• Fully satisfies (FS)
• Mostly satisfies (MS)
• Does not satisfy (DNS)

The final ranking of an LOI will be based on the final consensus rating.

Adjudicated elements

While all the information in the LOI proposal is to be considered, the following tables describe how three key supporting documents of the overall LOI submission correspond to the three criteria to be adjudicated.

Research program proposal

A research program proposal (or scientific strategy) will have one common strategic thematic area and can include multiple thrusts. The research program proposal must clearly demonstrate how these thrusts or sub-themes are linked and how they can be combined to provide a coherent strategic focus under an integrated institutional leadership and governance structure. In its proposal, the institution must make the case for the choice of theme and sub-themes and must describe how advancing these would serve the institution’s overall strategic focus for global excellence. Institutions were advised that submitting a scientific strategy with multiple sub-themes, some of which do not meet the standards of excellence expected for the CFREF, would weaken the overall competitiveness of the proposal. This could raise doubts about the institution’s ability to self-assess its global position and to establish objectives aligned with its demonstrated strengths (as assessed under criterion 1). It could also reveal a weakness in the institution’s ability to make the difficult choices that will undoubtedly be required to successfully implement an initiative of the CFREF’s scale and scope.

Equity, diversity and inclusion plan

Previously funded CFREF initiatives, if applicable

Budget requests

The LOI included a high-level budget request. A budget justification was not required at that stage. The purpose of the LOI funding request is to get a sense of the scale of the institution’s request for CFREF funds and to gauge overall demand. The budget request also enables reviewers to see the planned distribution of the proposed investment across budget categories and assess whether this aligns well with their own appreciation of where an institution needs to ramp up its capacity and its activities. It may also be used to recommend a manageable number of proposals to invite to submit a full application while ensuring a healthy level of competition.

There is no maximum or minimum limit to the amount that an institution can request, nor is there a prescribed award size. CFREF grants have seven-year terms, starting in 2023. It is expected that the budget requested will allow the institution to achieve global leadership and impact in the proposed area of research.

The 2022 competition has a budget of approximately $1.4 billion over seven years.

Program staff will conduct an administrative review of the budget items. A list of allowable costs can be found here: Eligible and ineligible expenses.

Focus of selection

The LOI must present information to assess the institution's existing scientific capacity in the area that it proposes to advance in a scientific strategy at the application stage (part of criterion 1), as well as the alignment of the proposed research with the government’s ST&I research priority areas (part of criterion 2). The LOI must also present information on its EDI plan (part of criterion 3).

Selection will be based on whether an institution meets the threshold and standard for consideration: does the LOI appropriately respond to the CFREF program’s intended goals and ambition of having significant global impact in the proposed research area. Selection is based on three criteria: an LOI must meet a minimum rating of “Fully Satisfies” for criteria 1 and 3, noted above, and must align with the government’s ST&I research priority areas for criterion 2, as rated by the Review Panel.

Based on the Review Panel consensus ratings, LOIs will be sorted into two tiers:

1. Top tier—will include LOIs that received consensus ratings of FS/FSE for criteria 1 and 3 and "yes" for criterion 2
   • Top tier LOIs rated as FSE for criteria 1 and 3 and "yes" for criterion 2 will be recommended to submit a full proposal.
   • Top tier LOIs rated as FS/FSE for either criteria 1 or 3 and "yes" for criterion 2: To ensure a manageable number of proposals and a healthy level of competition, the panel will rank these LOIs and recommend how many to invite to submit a full proposal based on the selection criteria and requested budgets.
2. Lower tier—will include LOIs
   • Rated as FS/FSE for criteria 1 and 3 and "no" for criterion 2
   • Rated as MS/DNS for criteria 1 and/or 3, regardless of the rating for criterion 2

The LOI evaluation process, including the final results of the panel meeting, will be submitted to the Steering Committee for its approval.

Handling documents used in peer review

Review documents contain personal information as well as information that the unauthorized disclosure of could reasonably be expected to cause serious injury (such as prejudicial treatment or loss of reputation or competitive advantage) to an individual, organization or government. Therefore, these documents are subject to regulation under the Privacy Act, the Access to Information Act and the Policy on Government Security. Protocols must be followed to ensure that information contained in applications, internal and external reviews, and panel discussions remains strictly confidential. Improper or unauthorized collection, use, disclosure, retention and/or disposal of this information can result in a privacy breach. Please refer to the Guide on Handling Documents Used in Peer Review for further details.

Legal and ethical information

Responsible conduct of research

Canada’s federal research funding agencies—CIHR, NSERC and SSHRC—are committed to fostering and maintaining an environment that supports and promotes the responsible conduct of research. The Tri-Agency Framework: Responsible Conduct of Research sets out the responsibilities and corresponding policies for researchers, institutions and the agencies that, together, help support and promote a positive research environment.

Confidentiality

Members appointed to the panel must read and agree to the Conflict of Interest and Confidentiality Agreement for Review Committee Members, External Reviewers, and Observers describing expectations and requirements.

Privacy Act

Personal information refers to any information about an identifiable individual. Based on the Privacy Act, personal information provided by applicants must be used only for assessing applications, making funding decisions, and related uses describing applicants at the time that their personal information is collected. Members are reminded that the use or disclosure of this information for any other purpose is illegal. It is important for panel members to adhere strictly to the guidelines set out in the Conflict of Interest and Confidentiality Agreement for Review Committee Members, External Reviewers, and Observers.

Canadian Human Rights Act

The activities of CIHR, NSERC and SSHRC are subject to the Canadian Human Rights Act. The purpose of the Act is to extend the laws in Canada to give effect to the principle that all individuals should have opportunity equal with other individuals to make for themselves the lives that they are able and wish to have. They should also have their needs accommodated, consistent with their duties and obligations as members of society, without being hindered in or prevented from doing so by discriminatory practices based on race, national or ethnic origin, colour, religion, age, sex, sexual orientation, gender identity or expression, marital status, family status, genetic characteristics, disability or conviction for an offence for which a pardon has been granted or in respect of which a record suspension has been ordered.

Official Languages Act

All review panel members and CFREF program staff must be aware of their obligations and rights as legislated in the Official Languages Act.