A proposal for assessing the FAIRness of data in Trusted Digital Repositories

[etsoupra]

[//]: # "==Do not write above this line== Instructions for posting issues: (1) Review what is already there. Perhaps a comment to an existing issue would be more appropriate than opening a new one? (2) Write your post below using Markdown (as per https://guides.github.com/features/mastering-markdown/ ) or just plain text. (3) Don't worry about these introductory lines - you can leave or delete them, as they won't display anyway (you can check this via Preview). (4) Hit the 'Submit new issue' button. ==Write below this line==" (see the FAIR Data Principles)

1. Blog post by Emily Thomas on Data Impact Blog by UKDS
2. EUDAT Webinar by Peter Doorn and Ingrid Dillo
3. GO FAIR Metrics Group

[CaroleGoble]

Are the FAIR Data Principles fair? Dunning, Alastair; de Smaele, Madeleine; Böhmer, Jasmin https://zenodo.org/record/321423#.WX9Do1GQyvE

[etsoupra]

Thanks a lot for your comment :) Very interesting article indeed - we are exploring how the information included can help us to make further improvements on the tool we are working on.

[dr-shorthair]

Nice. I particularly like this emphasis: "the facets provide targets that will help ..."

[sjDCC]

Thanks for the references @etsoupra - really useful. I've seen Peter and Ingrid's webinar, but the information in your post and details of the GO-FAIR metrics group will help as we write this up.

I've just been playing around with the assessment tool and had a couple of questions about the scores applied in the Survey Routing Diagrams:

• Will all data get a score of 1 or above? If you follow the 'no' paths (e.g. no PID, no metadata, no user licence, proprietary formats etc) it seems the dataset will get a score of 1 at each stage. Did you decide against 0 scores?

• Are you considering applying the scores or weightings to metadata as well as data? The main difference in the accessibility criteria is whether the data is open, public access so I guess some sensitive data will never be able to get a high FAIR rating, even if all the metadata are accessible and machine-readable etc. I read the Accessibility criteria A1.2 (the protocol allows for an authentication and authorization procedure, where necessary) as allowing closed or restricted access datasets. It might be useful to find a path that allows for non-open data to score as highly e.g. by awarding scores for clear metadata and info on how to gain access.

• What kind of response have you been getting to rebranding the final R as a resultant score? Is this working well so far or do you think you'll change the approach in the GO-FAIR metrics group?

Thanks again for all the inputs. We'll definitely be including this in the report :+1:

[etsoupra]

Dear Sarah,

Thank you very much for your email and feedback.

With regards to your questions:

• Yes, it is correct that the dataset will always score 1 or above at each stage. We decided not to include 0 scores because we wanted to create criteria for each star level (i.e. 1, 2, 3…) as we had already created 5 levels of criteria. Therefore, to add a 0 component would add an additional 6th criteria, for which we did not see necessary, considering we had already the 5 levels. We also thought it was perhaps a little more motivational to receive one star even when a dataset has nothing compared to receiving a 0 score! So, in short, yes all datasets will always score 1 or above.
• We have recently received some comments considering this point from some external reviews, so are currently in the process of updating the tool based on this feedback. One of the main points which came up was this issue about sensitive data receiving a lower score just because there is restricted access, therefore we are considering a higher star level for this kind of data if they have rich metadata. Additionally, if the restricted dataset allows users to request access then it may be able to score higher.
• Our operationalisation seems to work well so far, however, we are in the process of testing and making adjustments and improvements on the tool so that may be something we will consider.

Thanks again for the comments. Looking forward to the report ☺

Best, Eleftheria Tsoupra & Emily Thomas

From: Sarah Jones notifications@github.com Reply-To: FAIR-Data-EG/consultation reply@reply.github.com Date: Sunday, 20 August 2017 at 14:39 To: FAIR-Data-EG/consultation consultation@noreply.github.com Cc: Eleftheria Tsoupra eleftheria.tsoupra@dans.knaw.nl, Mention mention@noreply.github.com Subject: Re: [FAIR-Data-EG/consultation] A proposal for assessing the FAIRness of data in Trusted Digital Repositories (#23)

Thanks for the references @etsouprahttps://github.com/etsoupra - really useful. I've seen Peter and Ingrid's webinar, but the information in your post and details of the GO-FAIR metrics group will help as we write this up.

I've just been playing around with the assessment tool and had a couple of questions about the scores applied in the Survey Routing Diagramshttps://github.com/FAIR-Data-EG/consultation/files/1237051/SurveyRoutingDiagrams.pdf:

• Will all data get a score of 1 or above? If you follow the 'no' paths (e.g. no PID, no metadata, no user licence, proprietary formats etc) it seems the dataset will get a score of 1 at each stage. Did you decide against 0 scores?
• Are you considering applying the scores or weightings to metadata as well as data? The main difference in the accessibility criteria is whether the data is open, public access so I guess some sensitive data will never be able to get a high FAIR rating, even if all the metadata are accessible and machine-readable etc. I read the Accessibility criteria A1.2 (the protocol allows for an authentication and authorization procedure, where necessary) as allowing closed or restricted access datasets. It might be useful to find a path that allows for non-open data to score as highly e.g. by awarding scores for clear metadata and info on how to gain access.
• What kind of response have you been getting to rebranding the final R as a resultant score? Is this working well so far or do you think you'll change the approach in the GO-FAIR metrics group?

Thanks again for all the inputs. We'll definitely be including this in the report 👍

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/FAIR-Data-EG/consultation/issues/23#issuecomment-323582513, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AZeeEQnP3FcNIEbOHlNXxM1KxmA89WEKks5saCkEgaJpZM4ONdNj.