add contentTypes for Nutil Web, Web Ilastik and Web Align

[lzehl]

(tools are in dev and not yet registered in KG)

ContentType schema:

• https://humanbrainproject.github.io/openMINDS/v3/ --> select core/DATA/ContentType
• https://github.com/HumanBrainProject/openMINDS_core/blob/v3/schemas/data/contentType.schema.tpl.json

required property: name recommended property: extension, as well as description AND/OR link to specification

Missing contentTypes (suggested names):

• ???

[Majpuc]

Hi @lzehl , I am now ready to provide info for the generation of JSONLD for WebAlign and WebWarp. Could you point me to the procedure? I was looking for the JSONLD of QuickNII for an example but could not find it in the core/DATA/ContentType location.

[lzehl]

@Majpuc this is great news. You just need to make sure that you are looking at the latest dev branch for the content types (currently: v4). Here a link to one of the QuickNII content types:

https://github.com/HumanBrainProject/openMINDS_core/blob/v4/instances/data/contentTypes/quicknii%2Bjson.jsonld

Let me know if you need more help to provide a respective JSON-LD for the content types for WebAlign and WebWarp.

[Majpuc]

Thanks! I guess it would be wise :) Do I use the template listed above?

[Majpuc]

@lzehl here are proposed content types for WebAlign and WebWarp { "@context": { "@vocab": "https://openminds.ebrains.eu/vocab/" }, "@id": "https://openminds.ebrains.eu/instances/contentTypes/application/vnd.webalign.waln", "@type": "https://openminds.ebrains.eu/core/ContentType", "fileExtension": [ ".waln" ], "name": "application/vnd.webalign.waln", "relatedMediaType": null, "synonym": [ "WebAlign waln file" ] } The waln file contains image registration information. This file type was created for WebAlign and don't exist elsewhere.

[Majpuc]

{ "@context": { "@vocab": "https://openminds.ebrains.eu/vocab/" }, "@id": "https://openminds.ebrains.eu/instances/contentTypes/application/vnd.webwarp.wwrp", "@type": "https://openminds.ebrains.eu/core/ContentType", "fileExtension": [ ".wwrp" ], "name": "application/vnd.webwarp.wwrp", "relatedMediaType": null, "synonym": [ "WebWarp wwrp file" ] } The wwrp file contains image registration information. This file type was created for WebWarp and don't exist elsewhere.

[Majpuc]

NutilWeb wil be replaced by a python library. More information about the content type will come later.

[lzehl]

@k-dominik could you please let us know if Web Ilastik has the same or different content types than the Ilastik (https://github.com/HumanBrainProject/openMINDS_core/pull/309)?

[k-dominik]

There is are at least a few different output formats that one would like to add. Out of my head for sure .dzi (deep zoom) and maybe .n5 and precomputed chunks.

cc @Tomaz-Vieira :)

[lzehl]

@k-dominik and @Tomaz-Vieira this means Web Ilastik has the same input/output formats than Ilastik and in addition some more output formats or are they replacing formats from Ilastik? (Since Web-Ilastik seems also not to be registered yet in EBRAINS as software and webservice its a bit difficult to judge for me)

Let's try to become directly more concrete here:

• image/vnd.web-ilastik.dzi (output format; I assume this is not a general dzi image file but that has specific web-ilastik elements? IF NOT it would be just image/vnd.microsoft.dzi ?)
• application/vnd.web-ilastik.PURPOSE+n5 (output format; I assume this refers to N5 format as base?)

[Tomaz-Vieira]

I can't find the official MIME types you are mentioning , but I'll try to precisely describe what we are doing. Webilastik is a bit different from classic desktop ilastik, but if we already have something for desktop ilastik I think we can reuse almost all of it.

The interactive GUI can handle the following Inputs:

• .ilp files, which are "project files", identical to those used in desktop ilastik;
• Neuroglancer Precomputed Chunks, .dzi, and zipped .dzi (.dzip) which are all interpreted as images, with no added semantics

The interactive GUI can also produce the .ilp files mentioned above, as an output.

Webilastik can also launch non-interactive jobs. Those are more flexible with regards to their inputs:

• .png, .jpg, .dzi, .dzip, .tiff, .n5 and neuroglancer precomputed chunks, all of which are interpreted as images, with no additional semantics.

These jobs can produce image files with 2 possible semantics, both of which are also produced by desktop ilastik (so if we had these semantics already listed somewhere, we should definitely use the same ones):

• Pixel Classification, which are multi-channel, f32 images where each channel's value is between 0.0 and 1.0 and represents the likelihood of a pixel belonging to a particular class.;
• Segmentation, which are uint8 RGB images where each pixel is either red (255,0,0) if that pixel belongs to an object, or black (0,0,0) if that pixel does not belong to an object. These are usually consumed by Nutil-web, further down the Quint workflow

These two kinds of outputs (Prediction Maps and Segmentation) can be produced in the following formats:

• .dzi, .dzip, .n5 or Neuroglancer Precomputed Chunks

[lzehl]

@Tomaz-Vieira indeed there is currently no official MIME type for DZI registered. But following the IANA.org naming convention it should look like the suggestion I've made.

I think I need a meeting with the two of you (@Tomaz-Vieira @k-dominik) to solve this quickly and without any misunderstanding. Would that be possible? (@k-dominik since you have all emails would you kindly set something up?)

[lzehl]

(note: this works also for webilastik) application/vnd.ilastik.project+hdf5 (input/output) (extension: .ilp)

application/vnd.webilastik.pixelclassification+n5 (input/output) application/vnd.webilastik.pixelclassification+neuroglancer.precomputed (input/output)

image/vnd.webilastik.segmentation+dzi (input/output) image/vnd.webilastik.segmentation+dzip (input/output) image/vnd.webilastik.segmentation+n5 (input/output) image/vnd.webilastik.segmentation+neuroglancer.precomputed (input/output)

[lzehl]

ilastik: application/vnd.ilastik.project+hdf5 (input/output) image/png (input/output) image/jpeg (input/output) image/tiff (input/output) any hdf5 images (input) any n5 images (input) image/vnd.ilastik+hdf5 (input/output) application/vnd.ilastik.object-features+hdf5 (input/output) application/vnd.ilastik.object-features+csv (input/output) image/vnd.ilastik.pixelclassification+hdf5 (input/output)

webilastik: application/vnd.ilastik.project+hdf5 (input/output) image/png (input/output) image/jpeg (input/output) image/tiff (input/output) any n5 images (input) image/vnd.ilastik.pixelclassification+n5 (input/output) image/vnd.ilastik.pixelclassification+neuroglancer.precomputed (input/output) image/vnd.ilastik.segmentation+dzi (input/output) image/vnd.ilastik.segmentation+dzip (input/output) image/vnd.ilastik.segmentation+n5 (input/output) image/vnd.ilastik.segmentation+neuroglancer.precomputed (input/output)

[lzehl]

The ilastik pixel classification file is multi-channel, f32 image where each channel's value is between 0.0 and 1.0 and represents the likelihood of a pixel belonging to a particular class.

[lzehl]

The ilastik segmentation file is a uint8 RGB image where each pixel is either red (255,0,0) if that pixel belongs to an object, or black (0,0,0) if that pixel does not belong to an object.

[Tomaz-Vieira]

The ilastik object-features CSV file is a table with a line for every detected object and a column for every feature (e.g.: size, position) measured on that object.

[Tomaz-Vieira]

"Project File" is an hdf5 file with an .ilp extension. It holds the current state of the ilastik application such as what workflow is being used, which images are open and what features to compute on those images.

[Tomaz-Vieira]

'Object features in HDF5" are HDf5 files with two entries: table and images. The images entry contain one image cutout for each object and one mask for each object, marking the pixels occupied by that object. The table entry is saved as a numpy structured array and holds the selected feature values for each object.

[lzehl]

@Tomaz-Vieira that means we should probably should have sticked to image_vnd.ilastik+x-hdf instead of using a generic image hdf5 image_x-hdf since the combination of image + feature table is very specific to ilastik?

[Tomaz-Vieira]

@Tomaz-Vieira that means we should probably should have sticked to image_vnd.ilastik+x-hdf instead of using a generic image hdf5 image_x-hdf since the combination of image + feature table is very specific to ilastik?

We need both the generic and the ilasik-specific formats. The hdf5 files used for input in ilastik are just generic images. But the hdf5 outputs produced by ilastik have special semantics , one of them being that special "features table". I would say that the "image cutouts" that I mentioned in the previous comment are still a part of the feature table: it is one cutout per line, and their meaning is very tied to the whole concept of the feature table.

[lzehl]

@Tomaz-Vieira thanks for the quick reply. I will add the missing content type for the specific ilastik hdf5 asap.

[UlrikeS91]

I'm closing this issue since I split it up into two to be able to keep track of the parts and also to summaries to status for each of them better.