search

LibraryCarpentry / Top-10-FAIR

Top 10 FAIR Data & Software Things
https://librarycarpentry.org/Top-10-FAIR/
Other
32 stars 4 forks source link

imaging: archiving the original data file? #17

Open jcolomb opened 5 years ago

jcolomb commented 5 years ago

Hi, Thanks for making this available.

I am struggling with the question of archiving .nd2 files and whether I can get all metadata in a .omime.fiff. I asked here: https://forum.image.sc/t/data-archiving-what-format-to-use-which-files-to-keep/28947/11

What is your advice, should we archive both the nd2 file and a omime.tiff export, is the .nd2 file enough, is the omime.tiff file enough? Does it depend on the experiment?

orchid00 commented 5 years ago

Hi @jcolomb this question is too specific for me to answer. Where you posted it is the best place to get an answer. I'll just advice to add omero https://forum.image.sc/tags/omero I will close this.

jcolomb commented 5 years ago

While the specificity of the .nd2 format is clear, the question is quite general: should the proprietary formatted data be archived or should only the open formatted data be. It is also related to BIDS, which consider the open formatted data as "rawdata" and the original file as "sourcedata".

I think the question should be asked, if not answered, in the Top-10-FAIR/imaging chapter, and would like it to be re-opened for this reason.

orchid00 commented 5 years ago

I like your point, I will re-open this issue for anyone who would like to add comments to it. My two cents to this thread:

  1. What I will do is save both the open formatted data and the original files (in proprietary format) as long as there is storage space to do so.
  2. In most cases it is in the ethics clearance that researchers need to save the original files (the data that hasn't been yet formatted/touched).

Maybe the confusion arises when the definition is changed from raw to source data. Also, in theory if you have the original files it should be possible to export those into an open format, but when you do not have access to open the original files due to a proprietary format then yes providing the open format is ideal. Keeping in mind that some open formats may loose some details of the original file (and that is case by case).

orchid00 commented 5 years ago

adding from the 10 FAIR for imaging:

Section 8 b. File formats - Most file formats are defined by the data producer (e.g. instrument or software), whenever possible you should try to convert data to formats that are publicly accessible.