search

Project-MONAI / tutorials

MONAI Tutorials
https://monai.io/started.html
Apache License 2.0
1.86k stars 683 forks source link

Index embedding of MAISI #1764

Closed Z7Gao closed 3 months ago

Z7Gao commented 3 months ago

I tried to write the unconditional sampler of MAISI myself and found the body region embeddings confusing. Since the body region index is currently obtained from a mask (get_body_region_index_from_mask), what should top_region_index_tensor and bottom_region_index_tensor be when doing unconditional sampling?

@guopengf @Can-Zhao

guopengf commented 3 months ago

Hi, the diffusion unet is trained to condition the body region and spacing. As explained in this tutorial, "The body regions are formatted as 4-dimensional one-hot vectors: the head and neck region is represented by [1,0,0,0], the chest region by [0,1,0,0], the abdomen region by [0,0,1,0], and the lower body region (below the abdomen) by [0,0,0,1]. " The top/bottom region index roughly reflects where the CT scan cut-off is. For example, when top_region_index_tensor is [1,0,0,0] and bottom_region_index_tensor is [0,0,1,0], the generated image should cover from the head neck region to the abdomen region.

Can-Zhao commented 3 months ago

Hi, the diffusion unet is trained to condition the body region and spacing. As explained in this tutorial, "The body regions are formatted as 4-dimensional one-hot vectors: the head and neck region is represented by [1,0,0,0], the chest region by [0,1,0,0], the abdomen region by [0,0,1,0], and the lower body region (below the abdomen) by [0,0,0,1]. " The top/bottom region index roughly reflects where the CT scan cut-off is. For example, when top_region_index_tensor is [1,0,0,0] and bottom_region_index_tensor is [0,0,1,0], the generated image should cover from the head neck region to the abdomen region.

@dongyang0122 These seems to be important parameters to explain for users as they do not exist in standard diffusion model. I was wondering if you would like to expand it in tutorial?