Unable to "Segment Middle Slice"

[pawel-gawron]

Hello,

I'm currently working with SAM2 plugin for Slicer in version 5.4.0. I was following the installation and usage instruction, but during segmenting the middle slice I occur error.

My configurartion: Slicer 5.4.0 Cuda 12.2 Nvidia driver 535 Ubuntu 22.04

Error in Slicer:

Traceback (most recent call last):
  File "/home/pawel/Downloads/Slicer-5.4.0-linux-amd64/lib/Python/lib/python3.9/site-packages/numpy/lib/npyio.py", line 465, in load
    return pickle.load(fid, **pickle_kwargs)
_pickle.UnpicklingError: invalid load key, '<'.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/pawel/Documents/MedSAMSlicer/slicer/SAM2/SAM2/SAM2.py", line 484, in getMiddleMask
    segmentation_mask = np.load(result_path, allow_pickle=True)['segs']
  File "/home/pawel/Downloads/Slicer-5.4.0-linux-amd64/lib/Python/lib/python3.9/site-packages/numpy/lib/npyio.py", line 467, in load
    raise pickle.UnpicklingError(
_pickle.UnpicklingError: Failed to interpret file '/tmp/tmpel36d8al/result.npz' as a pickle

Error in server.py script:

Traceback (most recent call last):
  File "/home/pawel/anaconda3/envs/SAMM2_tutorial/lib/python3.11/site-packages/flask/app.py", line 1498, in __call__
    return self.wsgi_app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawel/anaconda3/envs/SAMM2_tutorial/lib/python3.11/site-packages/flask/app.py", line 1476, in wsgi_app
    response = self.handle_exception(e)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawel/anaconda3/envs/SAMM2_tutorial/lib/python3.11/site-packages/flask/app.py", line 1473, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawel/anaconda3/envs/SAMM2_tutorial/lib/python3.11/site-packages/flask/app.py", line 882, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawel/anaconda3/envs/SAMM2_tutorial/lib/python3.11/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawel/anaconda3/envs/SAMM2_tutorial/lib/python3.11/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawel/Documents/MedSAMSlicer/server.py", line 42, in download_file
    return send_file(output_name, as_attachment=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawel/anaconda3/envs/SAMM2_tutorial/lib/python3.11/site-packages/flask/helpers.py", line 499, in send_file
    return werkzeug.utils.send_file(  # type: ignore[return-value]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawel/anaconda3/envs/SAMM2_tutorial/lib/python3.11/site-packages/werkzeug/utils.py", line 428, in send_file
    stat = os.stat(path)
           ^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/pawel/Documents/MedSAMSlicer/data/video/segs_tiny/img_data.npz'

File img_data.npz renders in ".../MedSAMSlicer/" path, when I copy the file to directory "/home/pawel/Documents/MedSAMSlicer/data/video/segs_tiny/" then in Slicer error occur:

Traceback (most recent call last):
  File "/home/pawel/Documents/MedSAMSlicer/slicer/SAM2/SAM2/SAM2.py", line 484, in getMiddleMask
    segmentation_mask = np.load(result_path, allow_pickle=True)['segs']
  File "/home/pawel/Downloads/Slicer-5.4.0-linux-amd64/lib/Python/lib/python3.9/site-packages/numpy/lib/npyio.py", line 263, in __getitem__
    raise KeyError(f"{key} is not a file in the archive")
KeyError: 'segs is not a file in the archive'

How can i deal with those errors?

[Adsy42]

having same issue

[tnl-o]

Which version of Python are you using?

[pawel-gawron]

I use Python 3.11 in conda env

[Adsy42]

Yes same here, Conda with python=3.11. Also another bug I noticed is the .npz file gets saved in the root of the project folder as supposed to data/video... sub folder as written in the code

[tnl-o]

Install 3D Slicer at the root of C:/ and try again.

[tnl-o]

Cuda 12.2

How did you manage to use CUDA kernels for calculations?

https://github.com/bowang-lab/MedSAMSlicer/issues/31

[warmmilk-sudo]

having same issue, win11，CUDA12.4, python3.11, slicer5.7.0

[JUNXYU]

have the same issue. window 11 3D slicer 5.6.2

3D Slicer Error Log is here: `Traceback (most recent call last): File "C:/PythonProject/MedSAMSlicer-SAM2/slicer/SAM2/SAM2/SAM2.py", line 484, in getMiddleMask segmentation_mask = np.load(result_path, allow_pickle=True)['segs'] File "C:******\AppData\Local\slicer.org\Slicer 5.6.2\lib\Python\Lib\site-packages\numpy\lib\npyio.py", line 263, in getitem raise KeyError(f"{key} is not a file in the archive") KeyError: 'segs is not a file in the archive'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:**\AppData\Local\slicer.org\Slicer 5.6.2\lib\Python\Lib\shutil.py", line 625, in _rmtree_unsafe os.unlink(fullname) PermissionError: [WinError 32] 另一个程序正在使用此文件，进程无法访问。: 'C:\**\AppData\Local\Temp\tmpw6_jj58e\result.npz'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:**\AppData\Local\slicer.org\Slicer 5.6.2\lib\Python\Lib\tempfile.py", line 805, in onerror _os.unlink(path) PermissionError: [WinError 32] 另一个程序正在使用此文件，进程无法访问。: 'C:\**\AppData\Local\Temp\tmpw6_jj58e\result.npz'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:/PythonProject/MedSAMSlicer-SAM2/slicer/SAM2/SAM2/SAM2.py", line 485, in getMiddleMask self.showSegmentation(segmentation_mask, set_middle_mask=True) File "C:**\AppData\Local\slicer.org\Slicer 5.6.2\lib\Python\Lib\tempfile.py", line 830, in exit self.cleanup() File "C:**\AppData\Local\slicer.org\Slicer 5.6.2\lib\Python\Lib\tempfile.py", line 834, in cleanup self._rmtree(self.name) File "C:**\AppData\Local\slicer.org\Slicer 5.6.2\lib\Python\Lib\tempfile.py", line 816, in _rmtree _shutil.rmtree(name, onerror=onerror) File "C:**\AppData\Local\slicer.org\Slicer 5.6.2\lib\Python\Lib\shutil.py", line 749, in rmtree return _rmtree_unsafe(path, onerror) File "C:**\AppData\Local\slicer.org\Slicer 5.6.2\lib\Python\Lib\shutil.py", line 627, in _rmtree_unsafe onerror(os.unlink, fullname, sys.exc_info()) File "C:**\AppData\Local\slicer.org\Slicer 5.6.2\lib\Python\Lib\tempfile.py", line 808, in onerror cls._rmtree(path) File "C:**\AppData\Local\slicer.org\Slicer 5.6.2\lib\Python\Lib\tempfile.py", line 816, in _rmtree _shutil.rmtree(name, onerror=onerror) File "C:**\AppData\Local\slicer.org\Slicer 5.6.2\lib\Python\Lib\shutil.py", line 749, in rmtree return _rmtree_unsafe(path, onerror) File "C:**\AppData\Local\slicer.org\Slicer 5.6.2\lib\Python\Lib\shutil.py", line 608, in _rmtree_unsafe onerror(os.scandir, path, sys.exc_info()) File "C:**\AppData\Local\slicer.org\Slicer 5.6.2\lib\Python\Lib\shutil.py", line 605, in _rmtree_unsafe with os.scandir(path) as scandir_it: NotADirectoryError: [WinError 267] 目录名称无效。: 'C:\**\AppData\Local\Temp\tmpw6_jj58e\result.npz'`

but I find the result.npz in "C:\**\AppData\Local\Temp\tmpw6_jj58e" but no “seg” in result.npz Keys in the downloaded result.npz: ['imgs', 'boxes', 'z_range'] Keys in the result.npz: ['imgs', 'boxes', 'z_range']

[pawel-gawron]

Install 3D Slicer at the root of C:/ and try again.

Im using ubuntu 22.04

[pawel-gawron]

Cuda 12.2

How did you manage to use CUDA kernels for calculations?

https://github.com/bowang-lab/MedSAMSlicer/issues/31

I have natively installed CUDA drivers and created a virtual environment, but everything under ubuntu

[JasonQAQ]

having same issue Slicer 5.6.2 Cuda 12.1 Nvidia driver 535 Ubuntu 22.04

[yubanmeiqin12138]

having same issue, win10，CUDA12.2, python3.11, slicer5.6.2

[zemmyang]

same issue - rocky linux 9.2, cuda 12.1, python3.10, slicer 5.6.2

[yiyouyachuming]

same issue - ubuntu 24.04tls, pip install -e .,python3.11, slicer 5.6.2

[rasakereh]

It is usually a sign of faulty environment setup. Things like old CUDA or failing to download the checkpoints (see README file). If you have the environment set up properly, can you run this command in MedSAMSlicer directory and share the output with me?

python infer_video_tiny_debug.py --img_path 'img_data.npz' --gts_path X --propagate N --checkpoint tiny

[rasakereh]

@JUNXYU your case seems different. would you please run this command and share the output?

python infer_video_tiny_debug.py --img_path 'img_data.npz' --gts_path 'gts.npz' --propagate Y --checkpoint tiny

[zemmyang]

python infer_video_tiny_debug.py --img_path 'img_data.npz' --gts_path X --propagate N --checkpoint tiny

What should be X? I got a "No such file or directory 'x'" error.

Side note, I had to install a few packages before I could run infer_video_tiny_debug: torch, torchvision, hydra-core, tqdm, but installing them did not fix the error.

[rasakereh]

'X' is null char for our script. Use uppercase X instead of lowercase please

[zemmyang]

I got this from that command:

/opt/Slicer-5.6.2/Extensions/MedSAM2/sam2/modeling/sam/transformer.py:22: UserWarning: Flash Attention is disabled as it requires a GPU with Ampere (8.0) CUDA capability.
  OLD_GPU, USE_FLASH_ATTN, MATH_KERNEL_ON = get_sdpa_settings()
['img_data.npz']
0it [00:00, ?it/s]infering img_data.npz
0it [00:00, ?it/s]
Traceback (most recent call last):
  File "/opt/Slicer-5.6.2/Extensions/MedSAM2/infer_video_tiny_debug.py", line 297, in <module>
    infer_3d(img_npz_file, gts_file, propagate)
  File "/home/NAME/.conda/envs/medsam2server/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/opt/Slicer-5.6.2/Extensions/MedSAM2/infer_video_tiny_debug.py", line 210, in infer_3d
    assert np.max(img_3D) < 256, f'input data should be in range [0, 255], but got {np.unique(img_3D)}'
AssertionError: input data should be in range [0, 255], but got [   0    1    2 ... 5936 5940 5981]

[rasakereh]

I see. I updated the file infer_video_tiny_debug.py in the repository. you can download it and replace the old one with it. Try again to see if you can segment your image and keep me in the loop!

[zemmyang]

infer_video_tiny_debug got me this

Traceback (most recent call last):
  File "/opt/Slicer-5.6.2/Extensions/MedSAM2/infer_video_tiny_debug.py", line 300, in <module>
    infer_3d(img_npz_file, gts_file, propagate)
  File "/home/tcml/.conda/envs/medsam2server/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/opt/Slicer-5.6.2/Extensions/MedSAM2/infer_video_tiny_debug.py", line 220, in infer_3d
    img_resized = resize_bicubic_cv2(img_3D, (D, 1024, 1024))
  File "/opt/Slicer-5.6.2/Extensions/MedSAM2/infer_video_tiny_debug.py", line 196, in resize_bicubic_cv2
    resized_array[i] = cv2.resize(array[i], (new_w, new_h), interpolation=cv2.INTER_CUBIC)
cv2.error: OpenCV(4.10.0) /io/opencv/modules/imgproc/src/resize.cpp:4030: error: (-215:Assertion failed) func != 0 in function 'resize'

and when I try segment middle slice, this was the error:

Traceback (most recent call last):
  File "/opt/Slicer-5.6.2/lib/Python/lib/python3.9/site-packages/numpy/lib/npyio.py", line 465, in load
    return pickle.load(fid, **pickle_kwargs)
_pickle.UnpicklingError: invalid load key, '<'.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/Slicer-5.6.2/Extensions/MedSAM2/slicer/SAM2/SAM2/SAM2.py", line 484, in getMiddleMask
    segmentation_mask = np.load(result_path, allow_pickle=True)['segs']
  File "/opt/Slicer-5.6.2/lib/Python/lib/python3.9/site-packages/numpy/lib/npyio.py", line 467, in load
    raise pickle.UnpicklingError(
_pickle.UnpicklingError: Failed to interpret file '/tmp/tmprw3f0awy/result.npz' as a pickle

[rasakereh]

I see! it is a wrong type cast problem. I updated the file can you try again?

[zemmyang]

It works now, but the segmentation is not what I expected to be, I'm just getting a very small region. There are other unrelated problems as well; I'll write them in a different issue.

[rasakereh]

I am closing this issue as the thread's problem is resolved. Can you send me your image file, if there is no privacy/IP concern, along with the ROI information (the approximate coordination of the box + start and end slice)? To do this you can click on Markups (icon with three star-like thing) from the toolbar and see the details.

My email: reza.asakereh@uhn.ca

[lzhangsh]

tried """python infer_video_tiny_debug.py --img_path 'img_data.npz' --gts_path X --propagate N --checkpoint tiny""" using the latest version of infer_video_tiny_debug.py, still had the openCV error. Here's what I got