Related to label deviating position (DeepEdit multi label app)

[SachidanandAlle]

Dear Sachidanand,

First of all I would like to thank and congratulate you for your tremendous work. AI for healthcare imaging has never been more accessible.

I am a medical physicist in radiotherapy department in Amsterdam. I am currently investigating the possibilities of deep learning based segmentation in adaptive radiotherapy using MRI images. I've been using/developing monailabel APP (deep_edit multilabel app) for a while and so far I'm very happy with it.

However, I have encountered a few issues and have not been able to resolve them and I hope you can help me with that;

• This app generates labels with a (default) deviating position ( x: -2, y: -2, z: -2). Maybe this is because of a transform or something else do you have an idea how I can solve this?

-I use this app to segment different organs in the abdomen. The results are very good for 8 of the 9 organs (dice > 0.95), but for the duodenum it is much lower. Is there an opportunity within monailabel to put more focus on improving a single structure.

• Is there a possibility to automatically export the created labels in Dicom-RT file structure.

Kind Regards

OB

[SachidanandAlle]

@diazandr3s can you help on the deepedit issue. Looks like some post-transform issue. Or may be the problem is already solved in latest main branch.

-I use this app to segment different organs in the abdomen. The results are very good for 8 of the 9 organs (dice > 0.95), but for the duodenum it is much lower. Is there an opportunity within monailabel to put more focus on improving a single structure. MONAI Label deepedit is one kind of example App. There can be multiple ways to solve certain use-cases. For example you can even train segmentation model per single structure or use other networks. Or use combinations of models (not just one deepedit/deeplearning model)..

If you have DICOM Webserver, we currently support NIFTI/Label masks to get converted into DICOM-Seg using dcmqi tools (itkimage2segimage). We didn't see a use-case of producing DICOM-RT so far.

[diazandr3s]

Thanks for pinging me here. Regarding the post transforms, it was solved in the latest branch (https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/deepedit_multilabel/lib/infer.py#L161).

Would it be possible to try with the latest MONAI Label version? Install the weekly version: pip install monailabel-weekly and let us know how that goes :)

[OB-1606]

**_MONAI Label deepedit is one kind of example App. There can be multiple ways to solve certain use-cases. For example you can even train segmentation model per single structure or use other networks. Or use combinations of models (not just one deepedit/deeplearning model)._ <-- Thank you this is clear

If you have DICOM Webserver, we currently support NIFTI/Label masks to get converted into DICOM-Seg using dcmqi tools (itkimage2segimage). We didn't see a use-case of producing DICOM-RT so far. <-- I have used OHIF, you can directly import dicom files, but after segmenting there is only the option to export NRRD or NIFTI. Most systems in Radiation oncology work with DICOM-RT files it would be nice if there was a Dicom-RT directly.

Thank you for your quick response and help

[SachidanandAlle]

when you save/submit the label to monailabel.. it gets added as DICOM SEG into your Orthanc/DICOM server.. NRRD or NIFTI is for local download (for debugging purpose)

[OB-1606]

thank you for your last suggestion,

I installed the latest weekly version using [pip install monailabel-weekly] however after installing I got the message "Successfully installed monailabel-weekly-0.4.dev2205". Is this correct? because the latest version I can find is monailabel-weekly 0.4.dev2208.

After installation I ran a new test; x and y direction still a difference of -2, however in the z direction it is now correct so no more deviation.

[OB-1606]

thank you for your last suggestion,

I installed the latest weekly version using [pip install monailabel-weekly] however after installing I got the message "Successfully installed monailabel-weekly-0.4.dev2205". Is this correct? because the latest version I can find is monailabel-weekly 0.4.dev2208.

After installation I ran a new test; x and y direction still a difference of -2, however in the z direction it is now correct so no more deviation.

Is there other solutions to correct for the remaining problems

Kind regards

Verzonden vanuit Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 voor Windows

---

Van: SACHIDANAND ALLE @.> Verzonden: Wednesday, February 23, 2022 1:43:36 PM Aan: Project-MONAI/MONAILabel @.> CC: Bohoudi, O. @.>; Comment @.> Onderwerp: Re: [Project-MONAI/MONAILabel] Related to label deviating position (DeepEdit multi label app) (Issue #649)

when you save/submit the label to monailabel.. it gets added as DICOM SEG into your Orthanc/DICOM server.. NRRD or NIFTI is for local download (for debugging purpose)

— Reply to this email directly, view it on GitHubhttps://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FProject-MONAI%2FMONAILabel%2Fissues%2F649%23issuecomment-1048744108&data=04%7C01%7Co.bohoudi%40amsterdamumc.nl%7C80c8d7a721f94ce165e608d9f6ca1cd6%7C68dfab1a11bb4cc6beb528d756984fb6%7C0%7C0%7C637812170191865902%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=t5DN7XyCYyVA4QJ60%2BBbGQ9pPuobA%2FjlePgF3xGNHj4%3D&reserved=0, or unsubscribehttps://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFQQPXJSZEI4J3BTGYM7E6DU4TI7RANCNFSM5PECYBJA&data=04%7C01%7Co.bohoudi%40amsterdamumc.nl%7C80c8d7a721f94ce165e608d9f6ca1cd6%7C68dfab1a11bb4cc6beb528d756984fb6%7C0%7C0%7C637812170191865902%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=yBAS5nctR4Y1sdSCCbouu6c46StZfVnSNFwoc4TG2uI%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Co.bohoudi%40amsterdamumc.nl%7C80c8d7a721f94ce165e608d9f6ca1cd6%7C68dfab1a11bb4cc6beb528d756984fb6%7C0%7C0%7C637812170191865902%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=KL2FEqCX0xg5yfGiLG4TSfsJJXVjksd8EcDjYajVt50%3D&reserved=0 or Androidhttps://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Co.bohoudi%40amsterdamumc.nl%7C80c8d7a721f94ce165e608d9f6ca1cd6%7C68dfab1a11bb4cc6beb528d756984fb6%7C0%7C0%7C637812170191865902%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZVtK%2Bpc7OajMUv1REdC%2BLEY4HjiWS%2BJmmQuAIR3DTT4%3D&reserved=0. You are receiving this because you commented.Message ID: @.***>

---

VUmc disclaimer : www.vumc.nl/disclaimer AMC disclaimer : www.amc.nl/disclaimer

[diazandr3s]

Thanks, @OB-1606. That monailabel-weekly version should be OK. If you like, you could specifically install the latest version using this command: pip install monailabel-weekly==0.4.dev2208 With regards to the direction difference, would it be possible to see a print screen of the image/label? If you can, please also attach the log file (app.log) you can find in the logs folder.

[OB-1606]

Thanks,

Using --> pip install monailabel-weekly==0.4.dev2208 results in error:

[OB-1606]

regarding direction difference here a print screen:

[OB-1606]

hereby the log file

app.log

[diazandr3s]

Thanks for your quick response, @OB-1606. From what I can see, the predicted label is a bit blocky and moved because of the image resolution used to train the model.

This is the result of upsampling both image and predicted mask. By default, the deepedit_multilabel App resizes the images to 128x128x128. See here.

This means, images are downsampled before performing inference and then upsampled to the original size. The predicted label is originally 128 cube and then upsample to the original image size.

You can easily change this if you have enough GPU memory. What is the average image size you're using for this?

With regards to the monailabel version, let's use the one you have before (monailabel-weekly-0.4.dev2205) while we investigate the error you've faced. Don't think there are major differences between those.

[OB-1606]

Thanks again,

I checked for several images: 276 x 276 x 80.

I also use heuristic_planner does that matter?

[diazandr3s]

Yes, heuristic planner is working and it considers the available GPU memory to define the image size for training and inferencing. I take this from the log you attached before:

[2022-02-23 03:56:34,848] [9616] [MainThread] [INFO] (monailabel.utils.others.planner:71) - Available GPU memory: {0: 8785} in MB [2022-02-23 03:56:34,849] [9616] [MainThread] [INFO] (monailabel.utils.others.generic:131) - Using nvidia-smi command [2022-02-23 03:56:34,880] [9616] [MainThread] [INFO] (monailabel.utils.others.planner:75) - Spacing: [2. 2. 3.]; Spatial Size: [128, 128, 128]

I see your GPU has around 9 GB of memory available. I'm afraid it won't be possible to train a model with bigger image size using this GPU card.

[OB-1606]

so if I use a different GPU (larger memory size) solves this problem?

[diazandr3s]

Yes, it should! You can either leave the heuristic planner to decide the image size or you just disable it and fix the image size yourself here https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/deepedit_multilabel/main.py#L57

[SachidanandAlle]

based on the image and bug description.. this looks like an issue with resize (round) issue.. just a hint from myside.. may be i am wrong..

[OB-1606]

changing current GPU is not straight forward at this moment and I still cant understand why its always the same deviation?

[diazandr3s]

@SachidanandAlle, I think the same. The prediction is blocky and deviated because of the image resizing done for training and inference. One thing you could do @OB-1606 is label refinement after getting the prediction. This can be done using the manual tools available in Slicer.

[OB-1606]

what is the minimum GPU memory required to solve this?

I'm already using some post labeling steps which makes the labels smoother but the deviation makes it a bit confusing

[diazandr3s]

This is a good question, @OB-1606. This depends on different factors such as image size, data transforms, network architecture, batch size, etc Just as a reference, I've trained the multilabel deepedit using the BTCV dataset using an image size of 256x256x128, bacth size =1 and it occupies around 17 GB of GPU memory. As the images you're using are 276 x 276 x 80, my recommendation is to try an image size closer to that (i.e., 256x256x128)

Hope this helps.

[SachidanandAlle]

I see no further comments.. closing the issue for now.