search

heshuting555 / DsHmp

[CVPR-2024] Decoupling Static and Hierarchical Motion Perception for Referring Video Segmentation
76 stars 0 forks source link

Question about the training memory? Out of memory on 6 3090 cards #10

Open Raojiyong opened 1 month ago

Raojiyong commented 1 month ago

I have trained the model followed by the steps in readme, but I always have an out of memory error.

截屏2024-09-08 20 21 06

The only differences is that I'm training on 6 cards. So I replace the hyperparameters SOLVER.IMS_PER_BATCH and DATALOADER.NUM_WORKERS.

image

I wonder why this happened? Thanks!

heshuting555 commented 1 month ago

Thank you for your interest in our work! I have no idea about OOM.

It only needs about 19G memory. Maybe you can try to reduce the learning rate.

Raojiyong commented 1 month ago

Thanks for your reply!!

That's weird. Could you tell me the specific version you are running? I'm using python 3.8 with pytorch 1.9.1 and torchvision 0.10.1.

Sjunshu commented 1 month ago

Hello, can you solve it

Raojiyong commented 1 month ago

Hello, can you solve it

No, I haven't solved it