`modelMatrix` for loaders

[ilan-gold]

User story Right now, the OMETIFF loaders return a custom object that is used internally on the 3D layer and the scale bar layer for resizing/rendering physical size-informed data:

{
     x: {
        size: Pixels.PhysicalSizeX,
        unit: Pixels.PhysicalSizeXUnit
      },
      y: {
        size: Pixels.PhysicalSizeY,
        unit: Pixels.PhysicalSizeYUnit
      },
      z: {
        size: Pixels.PhysicalSizeZ,
        unit: Pixels.PhysicalSizeZUnit
      }
}

with NGFF v0.4/5 transforms spec and a more general need to handle non-square pixels in OMETIFF, it probably makes more sense to return a model matrix which can then be converted into a scale bar or be applied directly to the layer instead of having this handled internally by the layers, which means we need to track this information as well.

Preferred solution loaders now have a meta key for modelMatrix that allows for general transformations of the data. This will be resolved internally and by default be an identity matrix. Users can then choose whether or not to pass this down to the layers. This will be a fairly large breaking change for library users, but can be mitigated for upstream users of Avivator/Vitessce by implementing this behavior there.

Possible alternatives I think this is really the way to go. The alternative would probably be to convert everything to the new OMENGFF spec for transforms and then use that internally, but considering that it is currently under development, I do not think this is a good idea.

[manzt]

I think this is really the way to go. The alternative would probably be to convert everything to the new OMENGFF spec for transforms and then use that internally, but considering that it is currently under development, I do not think this is a good idea.

Agreed. Doesn't make much sense to me to translate to a different metadata model, to then translate back to a transformation matrix. Also, Viv doesn't only work with microscopy images but also TIFFs and Zarr datasets, and I don't think our loaders should enforce these use cases/users to figure out how to expressed a desired affine transformation with an OME data model.

[manzt]

Will the layers still take a modelMatrix as a prop? I think it would make sense to have a priority order of

modelMatrix passed as a prop > modelMatrix from loader if available > identity

[ilan-gold]

Will the layers still take a modelMatrix as a prop? I think it would make sense to have a priority order of

modelMatrix passed as a prop > modelMatrix from loader if available > identity

Right so this is an open question. Do we even want the layers checking the loaders for model matrices? If so, how do we decide the order of application? My instict was to have the layers do nothing, but maybe checking wouldn't be so bad. I would need to think about this. I think the order you have is correct, though. We probably don't want people using both, but I am unsure, which is why my default idea is "do nothing." I guess that's lazy, though, and we should probably ask the NGFF folks because it would be nice to have some default behavior.