bowang-lab / MedSAMSlicer

3D Slicer Plugin for Segment anything in medical images
https://www.nature.com/articles/s41467-024-44824-z
175 stars 20 forks source link

failed to use 3D Slicer plugin. #35

Closed pangshumao closed 2 months ago

pangshumao commented 2 months ago

Hi, I installed the MedSAM2 but failed to use the 3D Slicer plugin. I would appreciate you if you can help me to solve it. The installation and usage steps are as follows:

  1. SAM2 installation.

step 1. Download the MedSAMSlicer-SAM2.zip to directory /mnt/nas/pangshumao/software at https://github.com/bowang-lab/MedSAMSlicer/tree/SAM2, and unzip the MedSAMSlicer-SAM2.zip.

step 2. Create a virtual environment. conda create --name SAM2 python=3.11 conda activate SAM2

step3. Install the package. cd MedSAMSlicer-SAM2 pip install -e .

step 4. Download checkpoints. cd checkpoints ./download_ckpts.sh

step 5. Run the backend. cd ../ python server.py

  1. 3D Slicer usage.

step 1. Open the 3D Slicer on the server. Slicer

step 2. Open extension wizard on the 3D Slicer UI.

step 3. Select plugin directory. Click the Select Extension button, and select the /mnt/nas/pangshumao/software/MedSAMSlicer-SAM2/slicer/SAM2 as plugin directory.

step 4. Open a nifti MR image by 3D Slicer, and switch the Modules to Segmentation->SAM2.

step 5. Set the IP Address and Port to 192.168.10.103 and 8080 respectively.

step 6. Set the Preprocessing Options to MR, and click the Apply buttion.

step 7. Set the ROI.

step 8. Set the Model Size to small, and click Segment Middle Slice button.

The following errors appeared in the backend:

(SAM2) pangshumao@A6000:/mnt/nas/pangshumao/software/MedSAMSlicer-SAM2$ python server.py

The following errors appeared in 3D Slicer.

(SAM2) pangshumao@A6000:~$ Slicer Switch to module: "Welcome" Switch to module: "SAM2" "Volume" Reader has successfully read the file "/mnt/nas/pangshumao/data/Spine_Segmentation_3D_Space/L4-L5/In/Nii/MR/case1.nii.gz" "[0.20s]" [-103.30268730240817, None] [-103.30268730240817, -85.70268704014784] data sent is: {'checkpoint': 'small', 'input': 'img_data.npz', 'gts': 'X', 'propagate': 'N'} Traceback (most recent call last): File "/mnt/nas/pangshumao/software/Slicer-5.6.2-linux-amd64/lib/Python/lib/python3.9/site-packages/numpy/lib/npyio.py", line 465, in load return pickle.load(fid, **pickle_kwargs) _pickle.UnpicklingError: invalid load key, '<'.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/mnt/nas/pangshumao/software/MedSAMSlicer-SAM2/slicer/SAM2/SAM2/SAM2.py", line 484, in getMiddleMask segmentation_mask = np.load(result_path, allow_pickle=True)['segs'] File "/mnt/nas/pangshumao/software/Slicer-5.6.2-linux-amd64/lib/Python/lib/python3.9/site-packages/numpy/lib/npyio.py", line 467, in load raise pickle.UnpicklingError( _pickle.UnpicklingError: Failed to interpret file '/tmp/tmp7u7qxq94/result.npz' as a pickle

Note that the SAM2 and 3D Slicer 5.6.2 were installed in the same server which has 4 A6000 GPUs.

yiyouyachuming commented 2 months ago

I had a similar problem File "/mnt/nas/pangshumao/software/anaconda3/envs/SAM2/lib/python3.11/site-packages/werkzeug/utils.py", line 428, in send_file stat = os.stat(path) ^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/mnt/nas/pangshumao/software/MedSAMSlicer-SAM2/data/video/segs_tiny/img_data.npz'

rasakereh commented 2 months ago

Thanks for sharing the issue. Would you please run the following command and share the output with me?

python infer_video_tiny_debug.py --img_path 'img_data.npz' --gts_path X --propagate N --checkpoint tiny

pangshumao commented 2 months ago

Thanks for sharing the issue. Would you please run the following command and share the output with me?

python infer_video_tiny_debug.py --img_path 'img_data.npz' --gts_path X --propagate N --checkpoint tiny

Hi, the aforementioned command caused the following errors: UserWarning: Flash attention kernel not used because: (Triggered internally at ../aten/src/ATen/native/transformers/cuda/sdp_utils.cpp:720.) RuntimeError: No available kernel. Aborting execution.

I solved the aforementioned issue by modifying MedSAMSlicer-SAM2/sam2/modeling/sam/transformer.py as follows:

  1. remove the following sentence: OLD_GPU, USE_FLASH_ATTN, MATH_KERNEL_ON = get_sdpa_settings()
  2. add the following sentence: USE_FLASH_ATTN = False MATH_KERNEL_ON = True OLD_GPU = True

After that, SAM2 and 3D Slicer plugin can work. Thank you for your help.