Feature Request: Medical 3D images

[renatobellotti]

Hi,

This tool looks great! Are there any plans to support annotating medical images?

Potential applications:

• Segmentation
• Classification (also multi-class)
• ROI in every slice
• ROI in 3D

Useful formats:

• DICOM
• Niffti

Cheers!

[makseq]

Hi,

This tool looks great! Are there any plans to support annotating medical images?

It depends on the popularity of this request :-) But it's already frequently asked.

Similar request: https://github.com/heartexlabs/label-studio/issues/504

[fcossio]

adding my vote to the popularity of this request :)

[makseq]

BTW you can import a task like this:

{"image": ["https://example.com/1.jpg", "https://example.com/2.jpg", "https://example.com/3.jpg"]} 

and it will be displayed as an image with multiple layers.

[emblondel]

I'm working with DICOM, and we use label studio for annotation, I'd be glad to participate in the effort to build this feature. I uinderstand it si on the roadmap, is there a group I can join to help ?

[makseq]

@emblondel How do you see it should work? As I understand DICOM is rather heavy format and one image can have a size about 1 Gb, is it correct? So, there should be some converter from DICOM => jpg, yes?

[emblondel]

@emblondel How do you see it should work? As I understand DICOM is rather heavy format and one image can have a size about 1 Gb, is it correct? So, there should be some converter from DICOM => jpg, yes?

@makseq Hi! It all depends on the image that's inside. I work with 1Mb images, but having a high definition MRI for example can significantly increase the size. I don't know the typical size of an MRI, but I think 1Gb would be attained when joining all slices of the 3D Volume (1 dicom = 1 slice, all dicoms being generally named 000n.DCM). Now a Dicom contains not only the image itself (and sure the image can be extracted into jpeg, tiff, or what have you) but it also contain the metadata about the patient, the modality used, etc. The way this metadata is organized follows a naming standard (or should follow it anyways ;) Note that this data couldalso be useful in the process of annotation. There are a few libraries out there to manipulate dicom, I use pydicom in python, but I'm sure there must be a good one in JS. Now there also are a few good open-source viewers out there, and maybe I'm going out in a limb here but maybe there would be a sway to encapsulate the viewer and have them exchange with labelstudio to annotate in reference to the image.

[fcossio]

@emblondel How do you see it should work? As I understand DICOM is rather heavy format and one image can have a size about 1 Gb, is it correct? So, there should be some converter from DICOM => jpg, yes?

I can also give some input here.

In the context of breast imaging, different modalities create files of different sizes. For example the mammograms with which I work are on the order of 20-30MB per image. For MRI, a single slice can be in the order of 1-3MB and a volume would have between 100 and 600 slices.

We worked around the issue of 3D annotations by creating MIPs in two different planes and then intersecting the annotations. I must say, this has been not easy, and quite constraining at the same time.

Currently, there is an international academic project requiring MRI annotations. Maybe it is hearexlabs perfect opportunity to ship this MRI annotation feature?

[rz1027]

I would also add my vote to this feature request.

This would facilitate radiologists work especially when creating medical imagery datasets.

[erinmikailstaples]

Adding another note — this came up in the community slack this morning!

Request from James B. for multidimensional image data starting with 3D, but adding in the possibility to evaluate 3D+channels+time

[jimmib78]

I would also like to add my support for this but suggest this extend beyond purely medical images to encompas multi-dimensional imaging in general. It would then be useful for a huge amount of scientific image annotation. Formats could extend to

• Tiff stacks
• HDF5

Easy option would be to use BIOFORMATS or something similar as this adds a large range of formats without the need for conversion. http://www.openmicroscopy.org/bio-formats/

[ksidiro]

I am voting for this. I would start by simply adding DICOM and Nifti formats

[renatobellotti]

I appreciate the support! :) The more I think about it, the more I think DICOM might be too complex to support all eventualities. For example, CT overrides are sometimes used in radiotherapy treatment to mitigate the effects of imaging artifacts caused by metallic implants. So perhaps it might be a good idea to limit the scope only to the basic DICOM features and throw warnings for files that use advanced features.

I am a computational scientist, not a software engineer, and I am not familiar with the label studio code. That being said, I would like to suggest the following priorities:

1. Adapt the code to be able to handle 3D volumes that is agnostic of the underlying file format.
2. Write a DICOM reader/writer for this 3D format. Metadata are of secondary interest and could easily be added by user scripts as a postprocessing step. For machine learning they are typically ignored anyway.
3. Add an overlay system to support multichannel data and/are 4D images.

Of course, this is just a suggestion. What do others think?

[DimIsaev]

How are things going with this now?