Open rly opened 1 year ago
To be clear, that extra color channel is not differentiating a separate 'optic channel' as we normally refer to it?
We've seen ophys data before encoded as an extra appended dimension or even RGB when in actuality it represented grayscale of a single optic stream
Especially w.r.t.
Optimally, there would be flexibility for the 4th color channel dimension to include more than just RGB as well, since we are often imaging a range of different channels that don't always clearly map to just RGB values.
is that not what we have right now, simply with a separate 3D imaging plane (which could be called a 'volume' in that case) for each separate color channel (which they even say doesn't have to be RGB and thus would require the wavelength info anyway)
I asked for clarification, but I think that yes, each color channel is a separate "optical channel".
Yes, we can have separate 3D imaging planes for each separate color channel. But if the x, y, and z dimensions are the same across imaging planes, as it seems to be here, it may be more explicit and efficient to store these data as a 4D array rather than slices of the 4D array. But that's minor and probably it's better to minimize changes.
It seems like we still need a data type to represent single (static / non-time-varying) images associated with multiple imaging planes. We could have a grayscale volumetric TwoPhotonSeries
with one time point, but we would need N TwoPhotonSeries
, one for each of the N imaging planes (N channels). That seems hacky.
From the requester:
Yes, I think 4D: XYZC would be the ideal datatype with flexibility to allow any number of channels (the number of channels will likely be between 3-10 for almost all purposes). In terms of metadata, for each channel we would have the floating point emission value in nanometers, the filter used for that channel which would be formatted as center and width: ie if we're doing tagRFP with emission at 561nm and a bandpass filter at 700nm with width 70nm we would need these three pieces of information in the metadata. We also would want to include what each channel is used for- ie panneuronal marker, pseudocolor green, etc - but that can probably just be included in the freeform text description.
These image volumes are currently stored as image stacks in .tiff files and are in the range of ~50-150 MB.
They would also like to see support for time series of these multichannel volumes (5D: TXYZC). These files would also be stored as stacks of .tiff files but could be on the order of 30-50 GB.
It seems like we still need a data type to represent single (static / non-time-varying) images associated with multiple imaging planes.
I too am seeing requests for this at the NeuroDataShare, but for grayscale. Particular to being ophys-associated (light sheet microscopy) so having additional ophys metadata is nice, hence my draw towards representing this using photon series + optical channels
So really we just need something like a PhotonImage that has ophys metadata and represents the equivalent of a PhotonSeries with a single frame, which is what I currently recommend for maximum metadata
They would also like to see support for time series of these multichannel volumes (5D: TXYZC)
So, again, isn't this currently possible by splitting the PhotonSeries over each color channel, which is done mainly because each imaging plane of each photon series has a different optical channel, and it's the optical channel that actually encodes that essential metadata about the emission (granted we could improve by including optional fields for the filter, which seems pretty common)
The problem of allowing 5D max dimensions in a photon series is the identifiability of the columns in a 4D series: TXYC vs. TXYZ, which is why we specifically assign optical channels as the 'color' dimension indirectly by encoding as different series and capping the dimensionality at 4D to mean TXYZ
These image volumes are currently stored as image stacks in .tiff files and are in the range of ~50-150 MB.
NeuroConv and ROIExtractors would really appreciate examples of these files, are they willing to share a small file?
Agreed on all points.
To be clear, the proposed solution for time series of multichannel volumes, is to have N TwoPhotonSeries
objects (TXYZ), one for each of the N imaging planes (N channels). The timestamps and ophys metadata would be the same across the objects, except for the OpticalChannel
and maybe a description.
NWB does not support 4D colored, static volumes, but volumetric imaging is becoming more popular in neurophysiology experiments.
Context:
Proposal:
Create a new
Volume
type and make newGrayscaleVolume
,RGBVolume
,MultiChannelVolume
types as subtypes. We could addVolumes
andVolumeReferences
as analogs toImages
andImageReferences
. Maybe that is too much redundancy, but a volume is not an image and I don’t think inheritance is appropriate there.I have reached out to the author about what metadata are required for the color channels and will report back here.