search

Project-MONAI / MONAILabel

MONAI Label is an intelligent open source image labeling and learning tool.
https://docs.monai.io/projects/label
Apache License 2.0
598 stars 193 forks source link

XNAT Integration #223

Closed SachidanandAlle closed 1 year ago

SachidanandAlle commented 3 years ago

Integrate monai label server with XNAT

hshuaib90 commented 2 years ago

I would be very keen to have this integration working. Do we know if there are particular hurdles or challenges with XNAT integration?

SachidanandAlle commented 2 years ago

We need some xnat community to help on developing this plugin.. otherwise MONAILabel server has all the rest apis (ingredients) to support any viewers.. we took an extra step of providing such 2 reference implementations (3Dslicer and vanilla OHIF) to showcase the same.. these python/js based examples should be enough for others to follow.

diazandr3s commented 2 years ago

I would be very keen to have this integration working. Do we know if there are particular hurdles or challenges with XNAT integration?

This will be a great use case, @hshuaib90. MONAI Label + OHIF + XNAT Happy to chat more about this. As @SachidanandAlle said, MONAI Label is ready to be integrated with XNAT. It has DICOMWeb support and there are several apps that can be used to facilitate medical image annotation. There is also interest in supporting the new OHIF v3. See here https://github.com/Project-MONAI/MONAILabel/issues/603

hshuaib90 commented 2 years ago

We have XNAT server set up at my hospital and I'd be happy to support early testing. I'm afraid I don't know XNAT well enough to contribute to the development - at least not yet.

diazandr3s commented 2 years ago

Thanks, @hshuaib90. This is a great idea. Pinging @mjorgecardoso who can also contribute to this conversation :)

pwrightkcl commented 2 years ago

I have got MONAI label running at KCH, using the DGX as server and data store, and a standard NHS PC with Slicer as client. Long term, I expect we will want to use XNAT and OHIF too.

SachidanandAlle commented 2 years ago

I have got MONAI label running at KCH, using the DGX as server and data store, and a standard NHS PC with Slicer as client. Long term, I expect we will want to use XNAT and OHIF too.

That's good.. lets c on XNAT+OHIF if something can happen in near future.. MONAILabel is an opensource project.. so lets hope to get some more developers from XNAT community pitching in to develop the required plugin at XNAT side to interact with MONAI Label server.

pwrightkcl commented 2 years ago

I understand the current implementation is to use DICOM web. How does that handle labels?

  1. How do you feed it existing labels?
  2. Where does it put labels that you submit?
SachidanandAlle commented 2 years ago

DICOM Seg is created and pushed via DICOM Web server (given all permissions are there to read/write for MONAILabel server)

jamesobutler commented 2 years ago

For some additional knowledge on this topic, 3D Slicer + XNAT has existed with https://github.com/NrgXnat/XNATSlicer however it had become unmaintained and was recently disabled from the Slicer extensions catalog after some cleaning up of old extensions in preparation for the Slicer 5.0 release. It could be enabled again if there are developers interested in supporting it again. See https://github.com/Slicer/ExtensionsIndex/blob/master/ARCHIVE/XNATSlicer.s4ext

Update: It has been reenabled https://github.com/Slicer/ExtensionsIndex/blob/5f5ac42bab1ca7b57f25b78b12d39a4c6d9cc1ee/XNATSlicer.s4ext

DrRadiohead2 commented 2 years ago

Hi. We have xnat ml set up on aws. We were confused about deploying AIAA Or using monai label. Monai label looks more promising however needs more clarity about how data is trained which more straightforward in XNAT ML. We are more interested in inference tools. Can you please guide us how to deploy inference tool.

Regards Dr Akshaykumar

diazandr3s commented 2 years ago

Hi @DrAkshaykumar,

Thanks for your comment.

MONAI Label / XNAT integration is in the roadmap. This integration will allow both inference and training and user authentication should permit specific users to trigger training.

Are you using the DICOM Web protocol? or do you fetch images from XNAT directly when using AIAA?

DrRadiohead2 commented 2 years ago

Currently we have xnat deployed on AWS. We don't have AIAA set up now. That's why we were thinking how to go about which one to go with.

On Mon, 23 May, 2022, 1:10 pm Andres Diaz-Pinto, @.***> wrote:

Hi @DrAkshaykumar https://github.com/DrAkshaykumar,

Thanks for your comment.

MONAI Label / XNAT integration is in the roadmap. This integration will allow both inference and training and user authentication should permit specific users to trigger training.

Are you using the DICOM Web protocol? or do you fetch images from XNAT directly when using AIAA?

— Reply to this email directly, view it on GitHub https://github.com/Project-MONAI/MONAILabel/issues/223#issuecomment-1134592815, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQRVYUSYH7CWCTNR7KHNUI3VLNYTFANCNFSM5BAJAGNA . You are receiving this because you were mentioned.Message ID: @.***>

amrshadid commented 2 years ago

Any update ?

DrRadiohead2 commented 2 years ago

I would also like to know.

Regards

On Wed, 19 Oct, 2022, 7:34 pm Amr Shadid, @.***> wrote:

Any update ?

— Reply to this email directly, view it on GitHub https://github.com/Project-MONAI/MONAILabel/issues/223#issuecomment-1284418804, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQRVYUVKVP6CD44EUKEINQLWEA5L3ANCNFSM5BAJAGNA . You are receiving this because you were mentioned.Message ID: @.***>

SachidanandAlle commented 1 year ago

Some of the development and integration is completed on XNAT side.
And there must be an early version of the plugin available. https://wiki.xnat.org/documentation/xnat-ohif-viewer/release-notes-126157097.html

Please follow more on the XNAT development forum.

diazandr3s commented 1 year ago

Some of the development and integration is completed on XNAT side. And there must be an early version of the plugin available. https://wiki.xnat.org/documentation/xnat-ohif-viewer/release-notes-126157097.html

Please follow more on the XNAT development forum.

@danieltudosiu

DrRadiohead2 commented 1 year ago

Hi, We are trying to integrate XNAT with monai label, so that the models can be trained and deployed in the same pipeline. Can someone please prepare a detailed step by step guide to help us integrate it. We have currently hosted XNAT on AWS. @diazandr3s please discuss in meeting next year. Regards

danieltudosiu commented 1 year ago

Hi @DrAkshaykumar,

If it is ok with you I will join as well the meeting as I am working closely with @diazandr3s and MONAI Label's integration with XNAT is party of my PostDoc's JD.

I think I will be one of the main points of contact on this issue in the next year.

Best regards,

Dan

SachidanandAlle commented 1 year ago

Adding @AHarouni who has helped and lead this xnat + monailabel integration in many ways

AHarouni commented 1 year ago

Hi All

Currently the xnat datastore in monai label along with xnat=ohif viewer V3.4 enables the user to run inference of images in the project very efficiently as monai label can directly access xnat data. I did send code to xnat-ohif developer to allowing saving of dicom seg back to xnat. once this code is merged you would be able to run batch inference from monai-label api to run inference on all patients in a project convert o dicom seg and push this segmentation to xnat so user can load this seg Another addition that might be useful is using Rapid reader in xnat. This is unrelated to monai label but you can create a new worklist of all the patients which just finished batch infer for a quick way for radiologist to review and add comments to the quality of the AI.

There is a minor issue of unifying the AI dicom seg name and who user can name the dicom seg they create so monai-label + xnat datastore can figure out what is unlabeled vs labeled by a radiologist vs labeled by the batch infer. One issue here is each series can have multiple annotations one for liver tumor, kidney tumor etc

For end to end you should be able to use the xnat jupyter hub feature to launch monai jupyter container where you have access to the data of a given project. I successfully integrated this after modifying monai container and adhere to Xnat jupyter expectations

Open items I have on my list:

Hope that helps clarifies current status and next steps.

AHarouni commented 1 year ago

Hi All

Currently the xnat datastore in monai label along with xnat=ohif viewer V3.4 enables the user to run inference of images in the project very efficiently as monai label can directly access xnat data. I did send code to xnat-ohif developer to allowing saving of dicom seg back to xnat. once this code is merged you would be able to run batch inference from monai-label api to run inference on all patients in a project convert o dicom seg and push this segmentation to xnat so user can load this seg Another addition that might be useful is using Rapid reader in xnat. This is unrelated to monai label but you can create a new worklist of all the patients which just finished batch infer for a quick way for radiologist to review and add comments to the quality of the AI.

There is a minor issue of unifying the AI dicom seg name and who user can name the dicom seg they create so monai-label + xnat datastore can figure out what is unlabeled vs labeled by a radiologist vs labeled by the batch infer. One issue here is each series can have multiple annotations one for liver tumor, kidney tumor etc

For end to end you should be able to use the xnat jupyter hub feature to launch monai jupyter container where you have access to the data of a given project. I successfully integrated this after modifying monai container and adhere to Xnat jupyter expectations

Open items I have on my list:

Hope that helps clarifies current status and next steps.

mariamonzon commented 5 days ago

is there any up to guide/tutorial/example on how to implement monai label with XNAT?

AHarouni commented 4 days ago

Hi @mariamonzon It seems like that part of the documentation was moved. I will try to help you out, please try to follow steps below

  1. Install xnat by following their documentation. You can simply use their docker compose to bring it up quickly i . Verify you have oxnat ohif working by downloading jar file for ohif plugin for xnat and place it in the plugin folder. ii. you need to restart xnat for changes to work
  2. Verify that you monai label can start normally with local file system.
  3. use below commands before starting monai label 
    export MONAI_LABEL_DATASTORE=xnat
export MONAI_LABEL_DATASTORE_USERNAME=admin  -->replace it if you changed the user 
export MONAI_LABEL_DATASTORE_PASSWORD=admin  -->replace it if you changed the default password 
export MONAI_LABEL_DATASTORE_ASSET_PATH=   --> place the path to the xnat data folder, so monai label can simply use that path to resolve images, other wise monai label will download data from xnat using http calls 
export MONAI_LABEL_DATASTORE_PROJECT=  --> if you want to restrict it to one folder 
export MONAI_LABEL_DATASTORE_CACHE_PATH=  --> assuming you are not sharing the same file system if you need to cache 
export MONAI_LABEL_DICOMWEB_CONVERT_TO_NIFTI=false
monailabel start_server --app apps/radiology --studies http://<xnat_ip>:80 --conf models segmentation_spleen
  4. verify monai label starts and the logs show it logged into xnat correctly
  5. now you need to configure xnat ohif to see monai label 
    USER_PASS="admin:admin"
AIAASERVERPORT="http://<monallabel_ip>:8000"
ROOTURL="xnat_ip:80"
!curl -u $USER_PASS -X PUT "$ROOTURL/xapi/ohifaiaa/servers" \
    -H "accept: */*" -H "Content-Type: application/json" \
    -d "[ \"$AIAASERVERPORT\"]"
  6. configure ohif UI i . now go to a patient, open it in ohif, ii. go to preference, experimental, check monai label. iii. click masks from the tool bar, you should see a new icon called monai label
  7. in ohif click the right panel "masks" you should see a panel form moanai label and your model should show up there

Above steps enable the manual triggering of model inference, I have been working on automatically trigger inference once a new study is receiver by xnat.

Hope that helps. let me know if you face issues