Can we load MONAI models

[RYG81]

Hey,

I am looking for some anatomical image generation, after some research I found this source https://catalog.ngc.nvidia.com/orgs/nvidia/teams/monaitoolkit/models/monai_wholebody_ct_segmentation

But not sure how to use them in comfyui. Can you guide if it can be used or not used.

[city96]

That seems to be a model to segment CT scans, if I'm reading it right? I don't think there's support for stuff like that in ComfyUI, since it's more of a hobbyist solution.

What's your intended use case, and how would you expect the output to work? A colored seg map like the example image from your link, a separate image for each segmented area, or something else?

[RYG81]

May be the link I shared is of segmenting part, however the MONAI project has capabilities of generating infinite medical image with accuracy. https://www.youtube.com/watch?v=YHTSdd8-bnc&t=1s&pp=ygUXbW9uYWkgZ2VuZXJhdGluZyBpbWFnZXM%3D There are multiple models here - https://monai.io/model-zoo.html (diffusion models) I want to generate images similar to MRI and CT Scan, which are quiet hard to generate in SDXL models So if there is a way to utilize this in comfy or any other way which is more artist friendly than dev friendly is what I am looking for.

[city96]

The only two models from that page that generate stuff are the "Brats mri axial slices generative diffusion" one, which only has brain MRI scans and "Mednist gan" which is just a small, low resolution GAN. That nvidia video talks about "RadImageGan" (a finetuned StyleGAN-XL model), but I can't actually find the model weights anywhere, not even on the nvidia website they show in the video. All I found was this random forum thread of someone else not being able to find them lol.

[RYG81]

OK, I will try find model source and come back. Thanks for your time.

[RYG81]

If this will be of any help? https://github.com/BMEII-AI/RadImageNet

[city96]

Sadly no, that model is meant for detecting (labeling) medical issues as far as I can tell.