3Dslicer fail to respond and VRAM issue

[zhangjingcode]

It's is really impressive work! However, my 3dslicer fail to respond and can't do anything after "Start Mask Sync" in Samm Base. My 3dslicer version is 5.2.2

[bingogome]

Thanks for trying it out. We are continuously improving it. This week has been hectic, but I will release a new version lastest this weekend.

Could you send a more detailed bug report? Or, could you send a demo video showing how you were using it? Please note issue #13 (also found in ReadMe, Known Issue section). This is a pending problem plaguing the extension. I hope a new version over the weekend will help you more.

[zhangjingcode]

Thanks for your reply! I run sam_server.py and output "Starting to Wait for Message". But here's nothing happen when I load image in 3dslicer, even made some points and start sync

[bingogome]

You would need to compute the embeddings of the image first in the "Initialization" tab, and wait for the embeddings finishing. Let me know if it doesn't work.

[anxingle]

I click the button "Compute Embedding", and the console show the info:

Initializing SAM server ...
SAM server initialized ...
Starting To Wait for Messages ...
Computing embedding...
Processing slc0...
slc0 shape: (421, 512, 3)

the last two line is debug info which I added(I just add some print to debug).

[bingogome]

Thanks for the debugging info. @anxingle Can you also attach the GPU information of your machine? We recently received a bug report from 4GB Nvidia GeForce RTX 3050Ti, even though on our test machine, the VRAM usage is at around 3GB.

[anxingle]

Thanks a millon for your reply. My GPU is RTX 3060 Laptop GPU, and GPU memory is 6143MB.

[bingogome]

98129af38eff0fa422b5b5842134e6dbe770b869 Thanks for your report. I pushed a new commit with exception catch. Could you please report back when you get time to test it out?

[anxingle]

ok. I update the latest code, then run , it show out of memory:

RuntimeError: CUDA out of memory. Tried to allocate 1024.00 MiB (GPU 0; 6.00 GiB total capacity; 4.60 GiB already allocated; 0 bytes free; 4.67 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

[bingogome]

Unfortunately, this is something we don't have a solution at the moment, if we want to keep the segmentation process local. This is essential for the real-time-ness. A future option could be using their web interface (not sure if they provide it or not), which will limit the speed.

VRAM usage is also relevant to data size.

[anxingle]

Get it. Thank you very much. I will try to decrease the CT scans.

[zhangjingcode]

If I don't have GPU, can this function work?

[bingogome]

If I don't have GPU, can this function work?

Unfortunately SAM depends on GPU to work. Theoretically we can make a CPU running model, but I don't know how much worth in it, given in medical image cases there are other tools that are computationally less-intensive (and probably better performance).

[bingogome]

Get it. Thank you very much. I will try to decrease the CT scans.

I doubt reducing scans would improve. Single image slice resolution is probably the key.

[zhangjingcode]

I bought a 4070, it is installed in my computer, I gonna test it!

[bingogome]

Closing the issue.

[bingogome]

The repository now supports the vit_b which is significantly smaller.