ptsefton
commented
1 year ago How about make the directory the dataset with a part that is outside of the directory
{
"@id": "Mirax2-Fluorescence-2",
"@type": "Dataset", <-- or Collection (where the @id would have to be #Mirax2-Fluorescence-2 or could be URI?)
"mainEntity": {"@id": "Mirax2-Fluorescence-2.mrxs",
"hasPart": [
{"@id": "Mirax2-Fluorescence-2/Index.dat"},
{"@id": "Mirax2-Fluorescence-2/Slidedat.ini"},
{"@id": "Mirax2-Fluorescence-2/Data0000.dat"},
{"@id": "Mirax2-Fluorescence-2.mrxs"},
...
]
}
stain
simleo
This issue is the outcome of a discussion with @ilveroluca after last Tuesday's Workflow Run RO-Crate meeting, where we started wondering how to represent secondary files (as in CWL's secondaryFiles) in a Workflow RO-Crate.
The actual use case that gave rise to the discussion was the representation of a Mirax image, which consists of:
.mrxs;The directory contains data files, an index file etc. In the CRS4 tissue/tumor prediction workflow, in order to have CWL pick up all these files, we're using
secondaryFiles. However, those files are not really secondary, especially the data files, which contain the actual image data. Rather, all files together contribute to the same multi-file input dataset. An example of a format with a similar layout is Zarr.The real question, then, is how to represent such a dataset in RO-Crate. In RO-Crate, a
Datasetmaps to a directory, while single files are represented byFile(alias forMediaObject). What about mixes of files and directories? One of the solutions we discussed is to recommend usinghasParton the main file:However, in RO-Crate
Filerepresents a single file, and the files listed inhasPartare not actually parts (byte chunks) ofMirax2-Fluorescence-2.mrxs, but rather of the same dataset that includesMirax2-Fluorescence-2.mrxs. Moreover, not all formats clearly identify a "main" file: In Zarr, for instance,.zattrsand.zarrayare both metadata files at the same level.Another option could be to change the RO-Crate spec so that
Datasetwould map to a mix of files and directories, rather than a single directory. This is encompassed by the schema.org definition, which is very general. However, such a change at this point where several profiles and software packages already exist would be very disrupting, especially for tools.Though I've used an imaging example, the problem of representing a mix of files and directories as a single entity is quite general, so I think RO-Crate should have an explicit recommendation for this. Using a nested crate seems overkill, and depending on the format there might not be a single containing directory for the metadata file.
In principle, one could use
CreativeWork, but it's probably too general. It would be hard for tools to identify a multi-file dataset as such. Add a custom e.g.CompositeDatasettype? Is there an existing type that could be a good fit instead? Collection? Anyone knows of existing attempts to represent such datasets in RO-Crate?Another problem is what to put under
@id, especially when there is no clearly identified "master" file, since all actual files would have to go underhasPart. Can internal references be used for data entities? E.g.: