search

Project-MONAI / MONAI

AI Toolkit for Healthcare Imaging
https://monai.io/
Apache License 2.0
5.64k stars 1.03k forks source link

Integrate AzureML functionality into Auto3dSeg #6054

Open peterhessey opened 1 year ago

peterhessey commented 1 year ago

Is your feature request related to a problem? Please describe. No.

Describe the solution you'd like We would like to enable the option to run Auto3dSeg training and inference in AzureML. This is as Auto3dSeg is a fantastic tool for clinical use-cases, but many of the users who could use this tool do not have the compute resources available to train and host the models.

Microsoft Research Cambridge have developed the hi-ml-azure package for running medical imaging related machine learning pipelines in AzureML. By integrating features of this package into Auto3dSeg we would be able to provide the option to run all training and inference in the cloud.

Additional context Here the milestones we have identified, each of which is a feature that we hope to include in Auto3dSeg with no impact to the existing local user experience:

Furthermore, I have begun work on this in my own fork and am hoping to open a PR for the first milestone above before the end of next week.

mingxin-zheng commented 1 year ago

Hi @peterhessey , thank you for the demonstration. If I understand it correctly, this function manages the job submission in the AzureML. Are there going to be utility functions to monitor the status of the work on the cloud?

peterhessey commented 1 year ago

Hi @mingxin-zheng! There is already some utility for this if the user so desires: by passing in wait_for_completion=True, the local process will wait for the AzureML job to complete and report the its status when the job ends (e.g failed/completed). The function also gives the user a URL for their job once it is submitted, where they can monitor it in the AzureML Web UI.

When I write some proper integration tests for this, I will definitely be utilising wait_for_completion to ensure the AzureML jobs are behaving as expected.

I hope that answers your question 😄