search

LinShan-Bin / OccNeRF

Code of "OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments".
Apache License 2.0
318 stars 18 forks source link

Lack of memory on the GPU during training #16

Open dwz138831 opened 8 months ago

dwz138831 commented 8 months ago

Hi, I'm reproducing with the latest version of the code on 8 A30 GPUs (24GB of memory each) and it shows CUDA out of memory at the beginning of the 1st epoch, what is the cause of this.

1 2
LinShan-Bin commented 7 months ago

To reproduce the full results, you need 80G memory. The baseline method (no semantics, contracted_coord = False and auxiliary_frame = False) can be reproduced with 24G memory.

ZiyangYan commented 7 months ago

To reproduce the full results, you need 80G memory. The baseline method (no semantics, contracted_coord = False and auxiliary_frame = False) can be reproduced with 24G memory.

Hi, I change the config as you mentioned, and run it in 4 24G A30 GPU, but still has OOM problem

LinShan-Bin commented 7 months ago

To reproduce the full results, you need 80G memory. The baseline method (no semantics, contracted_coord = False and auxiliary_frame = False) can be reproduced with 24G memory.

Hi, I change the config as you mentioned, and run it in 4 24G A30 GPU, but still has OOM problem

For contracted_coord = False, we recommend you use a voxel size of [16, 200, 200]. This keeps the resolution of the inner part and removes the contracted outer part. Though we use [24, 300, 300] in our experiments for fair comparisons, to save memory you have to cut down the voxel size.