NCP Flanders
choose your programme
Horizon Europe

Horizon Europe is the main Research and Innovation Funding Programme of the European Commission.

Horizon 2020

Horizon 2020 is the previous Funding Programme. There are no longer calls, but projects can run until 2024.

Digital Europe

Digital Europe is a Funding Programme focused on bringing digital technology to businesses, citizens and administrations.

Other EU Funding

There are many other relevant EU funding programmes, managed by several agencies and directorates.

Infosheet


Pilot on Blind evaluation

This infosheet provides an overview of key elements to take into account for calls that will be evaluated blindly.

Programmes Horizon Europe   HorizonEU L+F  

Published on | 1 year ago

Last updated on | 3 months ago

Author Do you have an additional question? Or spotted a mistake? Don't hesitate to contact me!
Image of Sarah Stroobants
Sarah Stroobants

sarah.stroobants@fwo.be

Aim of the pilot

With this pilot the Commission wants to tackle concerns that the evaluation process could be perceived as biased towards well-known organisations in countries with better performing Research and Innovation systems. The Commission indicates that a recent independent study has not revealed such a bias.

The pilot aims to identify whether the implementation of blind evaluation within the legal requirements and operational context creates any difficulties. If this is not the case, it might lead to a modified approach, with a greater use of blind evaluations, which could effectively mitigate the risk of real, potential or perceived reputational bias.

The recording of a Commission information webinar on the topic is available here. The slides used during this webinar can be found in attachment.

Calls concerned by the pilot

In the Horizon Europe Work Programme 2023-2024, all first-stage proposals of two-stage call topics will be evaluated blindly (except one call for Widening).

Consequences for applicants

The concept of blind evaluation requires that experts evaluators do not know the consortium structure and the applicant(s) involved. To accomplish this, the following safeguards are in place:

  • Part A of the proposal will not be transmitted to evaluators.  
  • If Part B of the proposal includes any identification of the applicant, the proposal will be declared inadmissible and will be rejected.

Thus, applicants submitting a proposal under the blind evaluation pilot must take into account a new admissibility criterion: they must not disclose their organisation names, acronyms, logos nor names of personnel in Part B of their first-stage application (see General Annexes A &E). If identification as listed here above is mentioned by applicants in the proposal’s Part B, the proposal will be inadmissible.

In addition, names of proposers or their organisations should not be potentially identifiable indirectly, via, for instance, links to web pages or through references to their role and previous experience. The proposal can include references to their own publications if there is no emphasis that the publication is authored by one or more of the proposers. The proposal will be declared inadmissible if its anonymity is not respected in the part B of the application. Information on the identity of participants should be included only in proposal Part A.

Updated Application Form

The template of the standard application form for stage 1 applications has been updated accordingly and now includes a dedicated page with guidance for blind evaluations and annotations in the template text to guide applicants. Note that this template serves only as an example. The actual forms, provided in the online submission system under the Funding and Tenders Portal, might differ from this example.

The following extract from the dedicated page with guidance for blind evaluations (taken from the template of the standard application form for stage 1 applications) provides concrete examples:

  • How to insert references to publications? The proposal can include references to participants’ own publications if there is no emphasis that the publication is authored by one or more of the proposers.
    For example, the following statement will not be admissible: ‘For climate impact, we will use greenhouse gas emission intensities, following a methodology developed previously by a project partner (Dalin et al.)’
    but the following would be ok: ‘For climate impact, we will use greenhouse gas emission intensities, following the methodology described in Dalin et al.’

  • Examples of statements resulting in inadmissible proposals:
    - ‘Most of project’s participants have been involved in the previous H2020 project, NANOCOM...’
    - ‘Partner 3 is the leading company in Spain for wind turbine installation’
    - ‘The consortium consists of leaders in the high tech industry, including the biggest in terms of capital
    constructor of micro chips ‘
    - ‘The coordinator organisation was the one who first introduced the concept of m-RNA in vaccines’
    - ‘The consortium consists of 2 research centres (including an international one based in Geneva) and the
    oldest university in Belgium’

In the different sections of the standard application form template, annotations have been added to remind applicants to pay special attention not to expose their identity. Herewith two examples (Excellence part):

  • 'Describe how your project goes beyond the state-of-the-art, and the extent the proposed work is
    ambitious. Indicate any exceptional ground-breaking R&I, novel concepts and approaches, new products,
    services or business and organisational models. Where relevant, illustrate the advance by referring to
    products and services already available on the market. Refer to any patent or publication search carried
    out.' => Commission annotation in the standard application form stage 1 template: The description of the above should be provided in general terms, to preserve the anonymity of the proposal.

  • 'Any national or international research and innovation activities whose results will feed into the project, and how that link will be established.' => Commission annotation in the standard application form stage 1 template: The point above refers to activities in general and not to those carried out by the participants. It should be nevertheless kept in mind that the description has to be in line with the anonymity of the proposal.

It is important to keep in mind that stage 1 will only assess the excellence and impact criteria, whereas the quality of the partners and of the consortium composition (including relevant previous publications, achievements & projects of the consortium partners) will be assessed only in stage 2. More details about the two-stage submission procedure are available in this dedicated infosheet.

Background - Legal basis

The legal basis for this pilot can be found in Article 28 of the Horizon Europe Regulation:

Article 28 Award criteria and selection

[…] 4. The Commission and other funding bodies shall take into account the possibility of a two-stage submission and evaluation procedure and where appropriate, anonymised proposals may be evaluated during the first stage of evaluation based on one or more of the award criteria referred to in paragraph 1.

Attachments

There is 1 attachment connected to this article. Attachments are only accessible for people with an account on the NCP Flanders website.

Sign in (if you already have an account) or sign up.

myOVERVIEW
- sign up for personalised information

We offer news and event updates, covering all domains and topics of Horizon Europe, Digital Europe & EDF (and occasionally, for ongoing projects, Horizon 2020).

Stay informed about what matters to you. By signing up, you can opt in for e-mail notifications and get access to a personalised dashboard that groups all news updates and event announcements in your domain(s).

Only for stakeholders located in Flanders

Testimonial

image of PBNv2 - a MSCA ITN in the field of automotive R&D

PBNv2 - a MSCA ITN in the field of automotive R&D

The Marie SkÅ‚odowska-Curie Action (MSCA) European Innovative Training Network “PBNv2 - Next generation Pass-By Noise approaches for new powertrain vehicles” started in May 2017. Their research has the shared objective of investigating the possibilities to decrease pass-by noise of vehicles.

The project is a collaboration between 17 research institutions and companies in the European automotive R&D and provides a learning environment for 14 PhD fellows. The Belgian partner is the Noise and Vibration Research Group of KU Leuven, and this project is one of the many Horizon 2020 MSCA Innovative Training Networks that the KU Leuven research group participates in.