search

Project-MONAI / monai-deploy-app-sdk

MONAI Deploy App SDK offers a framework and associated tools to design, develop and verify AI-driven applications in the healthcare imaging domain.
Apache License 2.0
91 stars 46 forks source link

[FEA] support of customized components in monai_bundle_inference_operator #367

Open mingxin-zheng opened 1 year ago

mingxin-zheng commented 1 year ago

Is your feature request related to a problem? Please describe.

Currently MONAI bundle allows user to define their components (class) and supports the initiation of the class from the bundle template during inference. We can possibly extend monai_bundle_inference_operator.py to better support these customized components

MMelQin commented 1 year ago

Thanks @mingxin-zheng.

As the MONAI Bundle expands its capability, the App SDK MONAI Bundle (automated) Inference operator will need to follow suite in order to support more use cases (as of now, only pre-processing, post-processing, and inferer attributes, besides metadata.json, are supported).

I see this enhancement of low to medium difficulty, especially after the load and save image logic is separated out from pre and post processing respectively.

mingxin-zheng commented 1 year ago

@MMelQin Thanks for the suggestion and evaluation. I will submit a PR for your review.

By looking at the code, I am confused by the way to get MONAI bundle config and files. MonaiBundleInferenceOperator expects the metadata.json in the extra folder in the zipped bundle package (TorchScript style?) https://github.com/Project-MONAI/monai-deploy-app-sdk/blob/6cdc09d6f1283e8e424bfe9492e5666352ebdd6f/monai/deploy/operators/monai_bundle_inference_operator.py#L94

But the MONAI bundles are in a different format (example).

Is there an example how MONAI deploy can deploy the current MONAI bundle in the model zoo?

MMelQin commented 1 year ago

@mingxin-zheng MONAI Bundle IS a TorchScript with extra config data in JSON.

The example application using MONAI Bundle is the Spleen Seg, and the model is from Model Zoo Spleen Bundle. I have notice the bundle itself has changed over time, but the format remains the same.

mingxin-zheng commented 1 year ago

@MMelQin Thanks for the confirmation. I started with the assumption that bundle isn't always TorchScript. By looking at the storage, it seems most has .ts in the model folder but there are a few that did not follow. For example: https://github.com/Project-MONAI/model-zoo/releases/download/hosting_storage_v1/renalStructures_UNEST_segmentation_v0.2.0.zip

Maybe we make deploy to adapt both (torchscript and non-torchscript)? @MMelQin @Nic-Ma

MMelQin commented 1 year ago

@MMelQin Thanks for the confirmation. I started with the assumption that bundle isn't always TorchScript. By looking at the storage, it seems most has .ts in the model folder but there are a few that did not follow. For example: https://github.com/Project-MONAI/model-zoo/releases/download/hosting_storage_v1/renalStructures_UNEST_segmentation_v0.2.0.zip

Maybe we make deploy to adapt both (torchscript and non-torchscript)? @MMelQin @Nic-Ma

That would be too arbitrary and was the exact reason why we picked the common and standard model archive format.

mingxin-zheng commented 1 year ago

Okay I see. I downloaded all the bundles and check which model actually HAS model.ts file. Here is a snapshot @MMelQin @Nic-Ma

I may close this ticket as the issue sounds different that how it was first framed...

dbericat commented 1 year ago

Expand support for more Bundles with additional attributes or components.

Some of them are custom and already supported for Core and Label.

Today we only support pre-transform, inference and post-transform.