jbms
commented
3 years ago I thought a bit about this feature, here are some thoughts:
-
If there are a large number of channels, it may be desirable to only download the channels actually required by the shader. But that would require some way to determine which channels are required; in general the shader could make data dependent accesses to channels, which would be tricky to figure out. Requiring the user to manually specify the list of required channels would avoid this problem but would be somewhat inconvenient; the one mitigating factor is that if all data access is through invlerp controls, then the channels are already indicated. Otherwise, the required channels could potentially be determined by:
- superficial text-based analysis of the shader code, e.g. checking for numeric literals within calls to getDataValue.
- dynamic analysis of shader code; execute the shader code, with the real UI control values but calls to getDataValue would just return e.g. 0. Then we would see which channels are accessed, and those would be the ones fetched. This could potentially be done using WebGL to execute the shader, or using the same JavaScript GLSL interpreter also required for #259. This would not work if the channel access is data dependent, but that may be a reasonable limitation.
Perhaps it would be sufficient initially to always download all channels, though I suspect very quickly someone would attempt to use this with an array with a large number of channels.
-
Then there is the question of how to support this during rendering:
- One option is to rechunk the data as it is downloaded, to combine all channels into a single chunk as far as Neuroglancer is concerned. That would mean the data is redownloaded when the set of required channels changes, but maybe that is okay. The advantage is the neuroglancer rendering code could remain mostly unchanged.
-
The other (more natural) option would be to retain the original chunking of the data, where each chunk is loaded as a separate texture. The total number of chunks that could be used at any time would be limited by the number of textures that can be bound (32 on most implementations) --- i.e. 32 channels if channel chunk size is 1. That limit might be more of an issue if there is no mechanism for determining the set of required channels.
However, this does make the rendering code more complicated, because it becomes necessary to check for whether multiple chunks are available in GPU memory rather than a single chunk.
will-moore
pattonw
fcollman
Hey Jeremy, in #238 you mentioned:
Support for channels spanning chunks would be useful for supporting ngff/ome-zarr. While the specification does not require channels be split, most of the current tooling does (bioformats2raw, ome-zarr).