search

luca-medeiros / lang-segment-anything

SAM with text prompt
Apache License 2.0
1.53k stars 167 forks source link

How to make it run on GPU/CUDA? #59

Open jwahnn opened 3 months ago

jwahnn commented 3 months ago

When I try to run the model, the entire process takes about 20-30 seconds.

import torch device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') print('Using device:', device)

Shows that I am using cuda, but I am expecting it to take less than 10 seconds on my V100 GPU.

t5862755 commented 3 months ago

same question

I just would like to ask which version everybody use for pytorch. I downloaded torch for 2.0 but it gets error. My GPU is 1080ti

skulshreshtha commented 3 months ago

I think the SAM part should be running on CUDA if you have it but the GroundingDINO part is not configured in the project to use the correct device. It uses CPU by default as the device argument to this function is not passed. https://github.com/luca-medeiros/lang-segment-anything/blob/831cdc10906a23aabe4591a4be06f4c989e4ee30/lang_sam/lang_sam.py#L89

https://github.com/luca-medeiros/lang-segment-anything/blob/831cdc10906a23aabe4591a4be06f4c989e4ee30/lang_sam/lang_sam.py#L24

vietvo89 commented 2 months ago

If you follow the instruction to set the right path echo 'export CUDA_HOME=/usr/local/cuda' >> ~/.bashrc as in https://github.com/IDEA-Research/GroundingDINO, you should be able to run on GPU. You may need to replace cuda with your cuda version e.g. cuda-11.8.