search

FAIR-Data-EG / consultation

A call for contributions to the report of the FAIR Data Expert Group
Other
16 stars 3 forks source link

How can the FAIR principles be expressed as measurable properties? #5

Open Daniel-Mietchen opened 7 years ago

Daniel-Mietchen commented 7 years ago

as per https://twitter.com/band/status/874266773796859904 .

band commented 7 years ago

Thinking a bit out loud here:

Measurable properties have four attributes:

  1. A name.
  2. A concise definition or description.
  3. A Scale specifying the units and uncertainty of the property measurement. A usable Scale may be a binary value like True or False.
  4. A Meter, or method of measurement that describes how the property measurement is obtained.

One way to answer this question is propose properties that can be measured for each of the principles (Findability, Accessibility, Interoperability, and Reusability).

For example, what does it mean for a dataset to be "Findable"? On the Internet one measure of the ability to be found might be the ranking in organic results of a web search using domain specific terms. With current technology this ranking can be found by automation. With some standardization tools can be available for repository managers to verify "Findability". A dataset can be marked "Findable" if information about its location, or the location of its landing (description) page, is listed in specified online catalogs, or if this information is known to be harvested by standard tools or services.

Daniel-Mietchen commented 7 years ago

Some pointers to quantifying FAIRness are in

Dunning, Alastair, de Smaele, Madeleine, & Böhmer, Jasmin. (2017, January 31). Are the FAIR Data Principles fair?. Zenodo. http://doi.org/10.5281/zenodo.321423

For background, see https://openworking.wordpress.com/2017/02/10/fair-principles-connecting-the-dots-for-the-idcc-2017/ .

sjDCC commented 7 years ago

And also Ingrid Dillo and Peter Doorn's work, see: https://planeurope.files.wordpress.com/2015/03/doorn-fair-interoperability-poznan-plan-e-april-2017.pdf

sjDCC commented 7 years ago

Adding a comment here that came in from Jamie Shiers of CERN by email since it's predominatly definitions related. Also relevant for the DMP strand of our work.

From: Jamie Shiers [mailto:Jamie.Shiers@cern.ch] Sent: 13 June 2017 06:52 To: Sarah Jones (HATII) Sarah.Jones@glasgow.ac.uk Subject: Re: [rda-datamanagplans] Call for Suggestions and Contributions on Implementing the FAIR Data Principles: EC Expert Group on Turning FAIR Data into Reality

Hi Sarah,

Apologies if I missed something obvious, but I think a pointer to the current EU FAIR DMP guidelines would help.

In my experience people often take the words behind the acronym and “guess” what they mean, without looking at the guidelines.

For example, at our DPHEP workshop in March (on the 2nd day) someone claimed that our data was already FAIR - you just had to read physics papers to find it(!). (On the other hand, CMS attributed themselves quite high marks according to the proposed 5-star scale).

IMHO the current guidelines could clarified a bit, e.g.

“Are data discoverable with metadata” -> “Are data discoverable with metadata and/or other mechanisms, such as navigation”.

The CERN Open Data portal starts with the latter.

CERN -> Start analysing -> Choose experiment (CMS) -> Choose collection.

There are other ways of “finding” HEP data, e.g. through the HEPData portal, that includes navigation by experiment (not so many) or by other criteria (centre of mass energy of the interactions, reaction type and other criteria, e.g.

Measurement of the tt¯tt¯ production cross section using events in the eμμ final state in pp collisions at s√=s= 13 TeV

It is not clear to me how this fits with the brief description in the guidelines, which could therefore be further extended to include “discipline-specific criteria”.

FAIR was also discussed at the recent e-IRG workshop in Malta http://e-irg.eu/workshop-2017-6-programme as well as at the EU summit yesterday in Brussels. https://ec.europa.eu/research/index.cfm?pg=events&eventcode=44D86060-FBA1-1BD1-9355822B162BB0EE

I know Simon was at the latter but it would be good to collect input from all these events too.

Cheers, Jamie

micheldumontier commented 7 years ago

Hi, I've been working on frameworks for FAIR metrics. Some of these are outlined in presentations such as this one: https://www.slideshare.net/micheldumontier/fair-principles-and-metrics-for-evaluation

I'm also involved in two other groups on FAIR Metrics: NIH Working Group on FAIR metrics: https://bd2kccc.org/working-groups/?v=commons&h=front GOFAIR: https://www.dtls.nl/fair-data/fair-metrics-group/

As one of the founders of FAIR, I would be happy to provide input regarding FAIR and FAIR metrics and methods for reporting.

m.

dr-shorthair commented 7 years ago

We elaborated on a number of elements of FAIR, particularly relating to useability, also dataset maintenance and trust metrics. See https://confluence.csiro.au/display/OZNOME/Data+ratings We have proposed metrics within the following categories: