Making commercial data FAIR

[dafnevk]

For the corporate networks project, together with @lbogaardt , we make use of a commercial database. We would like to make this data more FAIR.

The current situation is as follows: 1) The data used is bought from a commercial organization. I do not know the details of the contract, but only researchers at the UvA can make use of the raw data 2) The data is delivered as SQL dumps, with limited meta data (some meta data is available on their online interface to the database) 3) These SQL dumps are processed, modified and put in a MySQL database on a server at the UvA 4) The research group queries this database, and usually does further analysis on an aggregated dataset (e.g. figures per country or per city, instead of per firm).

A student-assistant soon start working on step 3) for new dumps, and we want to make a plan to do this in a way that makes the data more FAIR. This raises the following questions (and probably more):

• Can we apply all FAIR principles to this data?
• What meta data should we store, and what if this meta data is not available?
• How to name the tables and fields to increase inter-operability and reusability?
• How to separate code that describes provenance, from implementation details (such as creating indices)? Is it necessary to have this separation?
• What agreements should we make with the data provider? For example:
  • Can we share the meta data? (e.g. names and descriptions of fields)?
  • Can we share the aggregated data?
  • Can we share the scripts used to transform the data?

[c-martinez]

Level's of data fairness can be found here.

[c-martinez]

On the topic of provenance, look at Metric R1.2

[dafnevk]

Additional question: How can we take care of the F part of fair when the data is owned by a commercial party?

• Can we make a DOI for a dataset that we don't own, or should we ask the data owner to do so?
• Can you register a DOI at, for example, Zenodo without supplying the actual data? Instead, we want to provide a description of what steps to take to access the data.

[PatrickAerts]

I think the F can only be validly implemented through the Metadata. These should be in terms that one would use to search for data. Through the meta data one can ensure that someone can become aware of the very existence of the (data) source, even if that source itself cannot be accessed other than by taking extra steps (like signing an agreement, accepting a license and pay). I think for the F it does not matter much if the owner is commercial or not. All the more if you refer to the data that you actually were given by the owner. Best regards, Patrick

From: Dafne van Kuppevelt notifications@github.com Sent: donderdag 29 maart 2018 14:37 To: NLeSC/data-sig data-sig@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [NLeSC/data-sig] Making commercial data FAIR (#14)

Additional question: How can we take care of the F part of fair when the data is owned by a commercial party?

• Can we make a DOI for a dataset that we don't own, or should we ask the data owner to do so?
• Can you register a DOI at, for example, Zenodo without supplying the actual data? Instead, we want to provide a description of what steps to take to access the data.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/NLeSC/data-sig/issues/14#issuecomment-377221583, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AEVBG-nXDx96Jin-Iep2MtPJOm2a4vyQks5tjNVfgaJpZM4S3dQb.

[c-martinez]

I think Patrick is absolutely correct, for F metadata is enough.

Specifically thinking of FAIR metrics:

• [ ] FM-F1A (Identifier uniqueness) -- if you create a DOI for your (meta)data, then you comply.
• [ ] FM-F2 (Machine readable metadata) -- depends only on your metadata
• [ ] FM-F3 (Resource identifier in metadata) -- here it gets tricky; but if you have a IRI of where the data can be found (even if it is behind a pay-wall), then you should be ok
• [ ] FM-F4 (Indexed in searchable resource) -- can you find it through google?

So no need to give access to the actual data, as long as you can point to a process to get this data (even if the process is "email this person, give him/her money, pick up data on a usb stick at their offices".

[c-martinez]

@dafnevk , @lbogaardt -- did this answer your question? Can we close this issue?

[dafnevk]

Yes, for now we have enough information!