search

rohit901 / cooperative-foundational-models

Official code for our paper "Enhancing Novel Object Detection via Cooperative Foundational Models"
https://rohit901.github.io/coop-foundation-models/
MIT License
50 stars 4 forks source link

running inference on single image overloads gpu memory #5

Open shahyaan opened 7 months ago

shahyaan commented 7 months ago

Hi,

When trying to run inference on a test image using your script I get a "CUDA out of memory error". My image size is 640x480, and I have a GPU with 24GB memory. I'd appreciate it if you could help me resolve this. Thanks!

rohit901 commented 7 months ago

Hello,

Thanks a lot for your interest in our work.

I tested it out on a 24GB machine available to me, and unfortunately it does not run on it, and requires 40GB GPU machine to run.

However, I can suggest you to try removing one of the models like either GDINO or SAM, and see if it helps you?

In the main script, sam is initialized here: (you may also have to remove the usage of sam in other parts of the script in scripts/novel_object_detection/ground_dino_utils.py in the "inference_gdino" function too.

sam init: https://github.com/rohit901/cooperative-foundational-models/blob/fb45630c46d75f7a1856654665191aeddeb3eb98/scripts/novel_object_detection/main.py#L60

gdino init: https://github.com/rohit901/cooperative-foundational-models/blob/fb45630c46d75f7a1856654665191aeddeb3eb98/scripts/novel_object_detection/main.py#L51

shahyaan commented 7 months ago

Hi Rohit,

Thanks for the quick response. I tried what you suggested, but even disabling SAM and then also MaskRCNN models doesn't seem to help with GPU memory overruns.

rohit901 commented 7 months ago

Did you try disabling GDINO itself, and using the SRM scores or refinement from SAM as it is? Depending on your use-case, it may still be helpful to you?

Otherwise, could you please try to run it on a 40GB machine? I've tested the code on 1x A100 as well which has 40GB VRAM.