Rec. 11: Develop metrics to assess and certify data services

[sjDCC]

Certification schemes are needed to assess all components of the FAIR data ecosystem. Like CoreTrustSeal, these should address aspects of service management and sustainability, rather than being based solely on FAIR principles which are primarily articulated for data and objects.

• Building on the model of CoreTrustSeal, new certification schemes should be developed and refined by the community to assess and certify other core components needed in the FAIR data ecosystem, such as identifier services, standards and vocabularies. Stakeholders: Global coordination fora; Data services; Standards bodies.

• Formal registries of certified components are needed: these must be maintained primarily by the certifying organisation, but should also be communicated in community discovery registries such as Re3data and FAIRsharing. Stakeholders: Data services.

• Steps need to be taken to ensure that the organisations overseeing certification schemes are independent, trusted, sustainable and scalable. Stakeholders: Funders; Research communities.

[ChiaraGabella]

SIB position: We believe that measuring the bioinformatics resources in a formal way through FAIR metrics is not the best way to assess the quality and accessibility of the resources. It is though important that the resource developers show their awareness of the concept and try to implement the aspects of it which are not too costly in terms of effort.

[hollydawnmurray]

F1000 position: It would be worth enquiring with CoreTrustSeal if they have any intention or interest to expand to certifying data services.

[katerbow]

DFG position: With respect to the Recommendations 10 and 6 metrics are not seen as an ideal way to certify science and scientific infrastructure. Using a rather careful approach in close cooperation with the scientific communities (in order to ensure choosing the appropriate parameters) is essential to approach this topic, which may come up with some metric type means of data service assessment. However, it is vital to ensure not to pave the path to a “control and assessment-industry” ending up in an unwanted administrative (and financial) overhead.

[Drosophilic]

The view from FAIRsharing: We are already working with a number of groups to provide metrics of FAIRness. These will be available on our site for databases, policies and standards.

My personal view: We should be careful not to confuse FAIRness with 'worth' and sustainability, as some services may be very FAIR, but have less good long-term sustainability due to the domain they work in. They shouldn't be penalised for this as different domains operate very differently in terms of infrastructure support, requirements and interest.

[pkdoorn]

Thumbs up. Think in particular of a CTS for sustainable and FAIR software.

[mromanie]

ESO position See comments to Rec 06.

[bertocco]

INAF (astronomy) position: see #10 and the other "metrics" RECs.

[MSoareses]

On item 2 I would again point to @Scholix as one such registry and therefore add that publishers should be stakeholders here as they are fundamental to linking data to literature.

[aidanbudd]

ELIXIR-UK position:

FAIRsharing is also part of the ELIXIR Interoperability Platform, contributed by the ELIXIR-UK Node, and therefore we support its use and recommendation of this action plan.

[gtoneill]

There should not be a too strong focus on the certification of services which could, as has been mentioned, lead to an overly control and assessment situation and industry. It is not clear what the relation is to Recommendation 10 on Trusted Digital Repositories. If there is overlap perhaps merge?