search

ljwztc / CLIP-Driven-Universal-Model

[ICCV 2023] CLIP-Driven Universal Model; Rank first in MSD Competition.
Other
559 stars 66 forks source link

Inference doubts #86

Closed hari3100 closed 1 week ago

hari3100 commented 3 weeks ago

Thank you so much for such an amazing model. With such high capabilities.

I have a few doubts , I ran the pred_psuedo.py inference , following the repo instructions, like this : (universalmodel) D:\whole_body_segmentation\universal_model\CLIP-Driven-Universal-Model>python pred_pseudo.py --data_root_path D:/whole_body_segmentation/universal_model/data/input_dir --result_save_path D:/whole_body_segmentation/universal_model/data/output_dir Use pretrained weights. load 117 params into 117 test len 1 0%| | 0/1 [00:00<?, ?it/s]2024-08-22 13:09:29,460 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Spleen.nii.gz 2024-08-22 13:09:35,866 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Right Kidney.nii.gz 2024-08-22 13:09:41,968 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Left Kidney.nii.gz 2024-08-22 13:09:48,069 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Gall Bladder.nii.gz 2024-08-22 13:09:54,103 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Esophagus.nii.gz 2024-08-22 13:10:00,132 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Liver.nii.gz 2024-08-22 13:10:06,258 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Stomach.nii.gz 2024-08-22 13:10:12,313 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Aorta.nii.gz 2024-08-22 13:10:18,409 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Postcava.nii.gz 2024-08-22 13:10:24,452 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Portal Vein and Splenic Vein.nii.gz 2024-08-22 13:10:30,633 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Pancreas.nii.gz 2024-08-22 13:10:36,799 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Right Adrenal Gland.nii.gz 2024-08-22 13:10:42,981 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Left Adrenal Gland.nii.gz 2024-08-22 13:10:49,122 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Duodenum.nii.gz 2024-08-22 13:10:55,166 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Hepatic Vessel.nii.gz 2024-08-22 13:11:01,258 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Right Lung.nii.gz 2024-08-22 13:11:07,236 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Left Lung.nii.gz 2024-08-22 13:11:13,289 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Colon.nii.gz 2024-08-22 13:11:19,373 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Intestine.nii.gz 2024-08-22 13:11:25,386 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Rectum.nii.gz 2024-08-22 13:11:31,489 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Bladder.nii.gz 2024-08-22 13:11:37,550 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Prostate.nii.gz 2024-08-22 13:11:43,552 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Left Head of Femur.nii.gz 2024-08-22 13:11:49,610 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Right Head of Femur.nii.gz 2024-08-22 13:11:55,642 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Celiac Truck.nii.gz 2024-08-22 13:12:02,421 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Kidney Tumor.nii.gz 2024-08-22 13:12:08,442 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Liver Tumor.nii.gz ung_081_Liver Tumor.nii.gz 2024-08-22 13:12:14,520 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\l2024-08-22 13:12:14,520 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Pancreas Tumor.nii.gz 2024-08-22 13:12:20,628 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Hepatic Vessel Tumor.nii.gz 2024-08-22 13:12:26,711 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Lung Tumor.nii.gz 2024-08-22 13:12:32,790 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Colon Tumor.nii.gz 2024-08-22 13:12:38,903 INFO image_writer.py:190 - writing: D:\whole_body_segmentation\universal_model\data\output_dir\lung_081\lung_081_Kidney Cyst.nii.gz 100%|████████████████████████████████████████████| 1/1 [23:16<00:00, 1396.88s/it]

So as you can see it took more than 23 mins to run 1 whole nifti file of a chest region , it was expected as I'm running this on my CPU. My main questions are below:

  1. If I use a GPU , will that increase the quality of the output masks?...or will it only reduce the time to generate results?

  2. I noticed that the model produced the results for all the organs it could , that means as many as it could find....is there a way to constraint it from doing that ?..or at least stopping the code from saving all the organs it inferred?

  3. This is more like me wondering how i could go on and write a python .py code for the inference , where i use this model as I'm doing now , but not through the cmd line but from a python file , did i make sense ?

Also as I was looking at the pred_pseudo.py , and i was wondering if this commented out part is the solution to my question 2., but I didn't try anything as I didn't want to mess a code which at least working : image

Any guidance or help will be greatly appreciated. Once again thank you for this model and your patience while clearing my naive doubts.

@ljwztc , @MrGiovanni , maybe one of you can help me?...Thank you for your time.

ljwztc commented 2 weeks ago

Hi @hari3100

  1. reduce time
  2. https://github.com/ljwztc/CLIP-Driven-Universal-Model/blob/459c8d3f644b4230f3c18ea3fed9aecb1e690279/pred_pseudo.py#L42 you can change this line
  3. you are go on with pred_pseudo.py
hari3100 commented 1 week ago

Hi @hari3100

  1. reduce time
  2. https://github.com/ljwztc/CLIP-Driven-Universal-Model/blob/459c8d3f644b4230f3c18ea3fed9aecb1e690279/pred_pseudo.py#L42 you can change this line
  3. you are go on with pred_pseudo.py

Hi @ljwztc Thanks for your response , im able to get individual organs. There's one more thing I would like to know, that is , is there a way to check the size of each detected tumor , or organ , or any detection ?... I know there's a way to do it with voxel values , but don't know how to access it nor do i know the proper way .... could you please guide me ? Thank you so much for your time and effort !!

ljwztc commented 1 week ago

You can use ITK-SNAP to visualize the output and use the built-in tool to see the tumor size.

hari3100 commented 1 week ago

You can use ITK-SNAP to visualize the output and use the built-in tool to see the tumor size.

Hi, yes I have done that , but is there a way to do this using some python libraries?...thank you for your response !!