mobie / mobie.github.io

1 stars 3 forks source link

Add support for different file formats to spec #49

Closed constantinpape closed 3 years ago

constantinpape commented 3 years ago

Given that the ome.zarr format is progressing (https://github.com/ome/ngff/pull/46) and that we may eventually want to support OpenOrganelle data (cc @bogovicj), I think we should add support for data formats that don't have a bdv.xml representation to the spec.

Currently, we store the data location in the spec like this:

"imageDataLocations": {
  "fileSystem": "bdv-file-for-local-data.xml",
  "s3store": "bdv-file-for-s3-data.xml"}

Here, we assume that the values are (relative) file paths that point to a bdv.xml file that in turn contains either the path to a local n5/hdf5 file or the address for the s3 storage.

However, for ome.zarr we don't need an xml file, because all relevant metadata is stored in the zarr attributes. In addition, for data that is stored on s3 (or some other object store), we also don't need to have any corresponding file in the project, but instead need to store the address.

So I would propose to add a new field fileFormat to support this. Initially we would support bdv.xml and we can probably add ome.zarr very soon.

For example:

"fileFormat": "bdv.xml",
"imageDataLocations": {
  "fileSystem": "bdv-file-for-local-data.xml",
  "s3store": "bdv-file-for-s3-data.xml"}
"fileFormat": "ome.zarr",
"imageDataLocations": {
  "fileSystem": "some-relative-path.ome.zarr",
  "s3store": "https://s3.embl.de/some-bucket/some-name.ome.zarr"}

or (eventually) for OpenOrganelle:

"fileFormat": "openOrganelle",
"imageDataLocations": {
  "s3Store": "s3://janelia-cosem/jrc_macrophage-2/jrc_macrophage-2.n5"}
constantinpape commented 3 years ago

@tischi I would suggest this small cosmetic change: https://github.com/mobie/mobie-viewer-fiji/pull/367. Also, could you make a new MoBIE-beta release with these changes so that I can test the new data formats in the release?

tischi commented 3 years ago

done done