About stpls3d training error CUDA out of memory

[LinLin1031]

9

I encountered the same problem. But if I don't use cloud GPU, how should I solve this problem?

[ngoductuanlhp]

You could try setting fp16=True in the config file to reduce the memory. You can also try with a smaller backbone version first (like this isbnet_lightweight_scannetv2.yaml).

[LinLin1031]

Sorry, I tried your suggestion but it didn't seem to work.

The exact location where I get that error is at https://github.com/VinAIResearch/ISBNet/blob/ceba716982de956239b8a626717816b74f8409bf/isbnet/model/isbnet.py#L807

I think it's because the shape of the "coords_" variable is too large. So how do I fix this?

[ngoductuanlhp]

You could reduce the n_queries to 128 or 192 during training and set it to 256 during testing. Moreover, you can also freeze the backbone by removing the arg --trainall when running the tools/train.py script.