I'm trying to reproduce the code, but I'm running into problems, the CPU memory keeps growing during training, I'm wondering if this is normal? (I used 32GB of CPU memory and ended up at around 6000 itrers and the program was killed due to full CPU memory)
After my testing, it is caused by sample=next(train_dataloader) in train_nuscenes.py mian(), do you know the cause of increased CPU memory consumption here?
Hello aharley, first thanks for your great work.
I'm trying to reproduce the code, but I'm running into problems, the CPU memory keeps growing during training, I'm wondering if this is normal? (I used 32GB of CPU memory and ended up at around 6000 itrers and the program was killed due to full CPU memory)
Any tips or help is greatly appreciated!