search

cvat-ai / cvat

Annotate better with CVAT, the industry-leading data engine for machine learning. Used and trusted by teams at any scale, for data of any scale.
https://cvat.ai
MIT License
11.77k stars 2.88k forks source link

Multichannel image view controls #7090

Open kerowak opened 8 months ago

kerowak commented 8 months ago

Actions before raising this issue

Is your feature request related to a problem? Please describe.

First off, thank you so much for the gift of CVAT. The power and extensibility of the AI tools has made annotation a breeze, and centralizing our annotated data has been an immense boon to model development.

My lab uses CVAT primarily to annotate fluorescence microscopy data of cells. For a given field of cells, we image in grayscale for up to four fluorescent channels. Each of these channels will highlight different structures of interest, often with some degree of spatial overlap.

We prepare these images for annotation by coloring each grayscale image according to its emission peak, then composite the set of colored images together.

Due to spatial overlap and intensity variations across imaging channels, it can be sometimes be difficult to resolve certain structures in composited images during annotation. Often, dim structures in one channel will be obscured by bright structures in a spatially-overlapping channel.

I have seen bioimage-oriented annotation tools address this issue by providing per-channel intensity adjustment sliders that allow the user to adjust the image's appearance and focus on structures of interest.

To my knowledge, CVAT lacks these specific intensity adjustment controls.

Describe the solution you'd like

Please add per-channel intensity adjustment controls for multichannel images.

Such intensity adjustment controls typically take the form of a dual range slider with range 0-100. The lower and upper sliders represent the percentiles for an image rescale operation that shrinks/stretches the intensity range of the image in that specific color component.

Describe alternatives you've considered

To work around these limitations in instances of significant spatial overlap in multichannel images, I have foregone the compositing step and uploaded channel-stacks as movies. I then switch between channels in an annotation task by switching between movie frames. This solution works alright, but it doesn't work for timeseries imagery and makes it harder to visualize the relationship between channels in certain cases.

Additional context

No response

PierreBoyeau commented 6 months ago

I would also be very interested in this! @kerowak do you know existing annotation tools that already provide such a feature?

kerowak commented 6 months ago

@PierreBoyeau the Big Two that I'm thinking of are ImageJ and Napari

Obviously you lose the centralization/organization of CVAT when you use these tools, but if you only need to annotate a few projects the simpler workflow and multichannel controls may be the way to go.

Of the two I would pick Napari, as it has a segment anything plugin