Compatibility with bdv-playground xml format

[constantinpape]

@tischi @martinschorb @NicoKiaru Let's move the discussion about the MoBIE and bdv playground metadata here.

Specifically:

1. Is there a fixed definition of the additional xml fields for bdv playground somewhere?
2. For MoBIE, the metadata is stored as json files and is defined via jsonschema, see https://github.com/mobie/mobie.github.io/tree/master/schema. A description of the format is here: https://mobie.github.io/specs/mobie_spec.html. Please note that this is still work in progress.
3. @martinschorb your suggestion https://github.com/mobie/mobie.github.io/issues/26#issuecomment-808195972: yes, adding the bdv-playground metadata from the xmls if present sounds good. But note that I am currently rewriting the whole python library a bit to work with the new spec and to have a more consistent api, see https://github.com/mobie/mobie-utils-python/pull/21. The relevant functions now are https://github.com/mobie/mobie-utils-python/blob/spec-v-02/mobie/metadata/source_metadata.py#L38 and https://github.com/mobie/mobie-utils-python/blob/spec-v-02/mobie/metadata/view_metadata.py#L103

[martinschorb]

Hi,

I think it is defined as in

https://github.com/bigdataviewer/bigdataviewer-playground/blob/8563f16c0493603b47156bd8d15e0813cb798928/src/main/java/spimdata/util/DisplaysettingsHelper.java

looks like:

<Attributes name="displaysettings">
        <Displaysettings>
          <id>0</id>
          <isset>true</isset>
          <color>255 255 255 255</color>
          <min>0.0</min>
          <max>255.0</max>
          <Projection_Mode>Sum</Projection_Mode>
        </Displaysettings>
      </Attributes>

The color is specified as ARGB values. I wonder how we can map it to the ColorMap strings from MoBIE.

[martinschorb]

I found these base colors, but there are some more apparently. Where are these defined? https://docs.oracle.com/javase/7/docs/api/java/awt/Color.html

randomFromGlasbey is only used by MobIE (google finds nothing else...)

[constantinpape]

The color is specified as ARGB values. I wonder how we can map it to the ColorMap strings from MoBIE.

We support ARGB as well, see https://github.com/mobie/mobie.github.io/blob/master/schema/view.schema.json#L24-L27.

[NicoKiaru]

Hi @constantinpape,

1. there are the xml spec from bioformats : https://docs.openmicroscopy.org/bio-formats/6.6.0/formats/big-data-viewer.html (but I can't see the specs document). This relates to sources positionining (and voxel sizes) in 3d - affine transform 3d supported only. - this, we cannot modify

   We added a custom attribute in the xml file, which is currently defined in https://github.com/NicoKiaru/bigdataviewer-spimdata-extras/blob/master/src/main/java/spimdata/util/XmlIoDisplaysettings.java . These can be modified in order to store a json string reprensentation into the xml tag. No issue to modifiy this one for better compatibility

[martinschorb]

We support ARGB as well, see https://github.com/mobie/mobie.github.io/blob/master/schema/view.schema.json#L24-L27.

OK, cool. That makes the import of BDV-PG XMLs easy.

[constantinpape]

https://github.com/NicoKiaru/bigdataviewer-spimdata-extras/blob/master/src/main/java/spimdata/util/XmlIoDisplaysettings.java .

Ok, this should be straight forward to translate to our imageDisplay metadata:

• isset: not relevant for us, because source selection works a bit differently.
• color -> color
• min, max -> contrastLimits

[NicoKiaru]

* `isset`: not relevant for us, because source selection works a bit differently.

ìsset` is not a good name, I think it is false when the source/image is of type RGB, in which case the color is ignored

[constantinpape]

ìsset` is not a good name, I think it is false when the source/image is of type RGB, in which case the color is ignored

Ah ok, then I misinterpreted this. What exactly does it do?

[NicoKiaru]

Yes, it's lacking documentation.

The color field is used/applied if the image pixel is a single numerical value (=single channel). For ImgLib2, this means the underlying pixels are unsigned short, unsigned byte, etc.

But I also need to handle the case of images where the pixel type is argb - typically histology images which have been acquired with color cameras. In this case, there's no color which needs to be applied to pixels values - the original rgb value is displayed (ok, maybe with some min max rescaling). In this case the color field of display settings is ignored, and the isset flag should be set to false. If you don't have rgb images, you can just set the flag to false.

Again, we can change this spec. I don't have time to have a look now, but we can discuss this next week. I think it would be nice if we can already specify some layer index, even if it's not supported yet.

[tischi]

This is exciting guys! If we can make the two specs work well together that would be a big step forward! 🚀

[tischi]

But I also need to handle the case of images where the pixel type is argb - typically histology images which have been acquired with color cameras.

I think that's a good point. I think the integer encoded ARGBType is useful (could be also useful for ome.ngff to store RGB images without having to worry about adding an additional color dimension, ping @joshmoore ):

final public static int rgba( final int r, final int g, final int b, final int a )
{
    return ( ( r & 0xff ) << 16 ) | ( ( g & 0xff ) << 8 ) | ( b & 0xff ) | ( ( a & 0xff ) << 24 );
}

However, the data type currently is not specified in the bdv xml which means one cannot know whether this should be opened as normal integer image or as an ARGBType. What about adding a required field dataType to our specifications to cover this and potentially also other such cases where people want to encode something fancy into a single number? Also I think it is very convenient to be able to quickly inspect the dataType in the JSON or XML .

[constantinpape]

@NicoKiaru thanks for the clarification, makes sense now. I don't quite understand what you mean by "layer index" though:

I think it would be nice if we can already specify some layer index, even if it's not supported yet.

@tischi

What about adding a required field dataType to our specifications to cover this and potentially also other such cases where people want to encode something fancy into a single number? Also I think it is very convenient to be able to quickly inspect the dataType in the JSON or XML .

I like the idea, but I wouldn't use the name dataType. It could be confused with the actual data type (int, float, etc.). May be imageType instead?

[tischi]

imageType reminds me a bit more of the image vs mask vs segmentation distinction.

At least in ImgLib2, I think, ARGBType is conceptually the same as IntegerType or UnsignedByteType, which specifies the "thing" that you get when you ask the image to give you then content of one pixel, which I think could be called the dataType of the image.

[constantinpape]

I am happy with choosing something else than imageType, but dataType is really not a good choice; it's used as the field to indicate the data type in n5: https://github.com/zarr-developers/zarr_implementations/blob/master/data/n5-java.n5/gzip/attributes.json#L1

[joshmoore]

could be also useful for ome.ngff to store RGB images without having to worry about adding an additional color dimension, ping @joshmoore ):

Happy to open up the discussion again but if I'm understanding correctly (IIUC) I think I made the argument to not include specialized types and had at least some supporters on an NGFF call (I'd have to check the notes). The reasoning was that the (a)rgb logic in, e.g., Bio-Formats can already get fairly complicated, and now we have a ND spec and can make use of it.

[tischi]

The reasoning was that the (a)rgb logic in, e.g., Bio-Formats can already get fairly complicated, and now we have a ND spec and can make use of it

@joshmoore OK! Just wanted to ping you in case it was useful. Let's see how the ND NGFF spec develops for ARGB sort of images!

[tischi]

@constantinpape what about pixelType?

[constantinpape]

I think pixelType is good.