search

MediaBrain-SJTU / LED

[CVPR2023] Leapfrog Diffusion Model for Stochastic Trajectory Prediction
138 stars 17 forks source link

Out of memory issue while running ''def _test_single_epoch(self):'' function #3

Open daviddaiiiii opened 1 year ago

daviddaiiiii commented 1 year ago

Hi there, I'm currently trying to reproduce the training and evaluation process but have run into an issue with the "def _test_single_epoch(self):" function causing an OOM error.

I have this issue running on a 24G RTX4090 GPU, it seems that this is not due to hardware limitations.

Do you have any suggestions on how to modify the function and release some memory during the evaluation?

######################################################## Trying to train from scratch image

######################################################## modify image get image confirm the issue is due to ''def _test_single_epoch(self):'' function ######################################################## solely evaluating with pre-trained weight image