search

USTC-Video-Understanding / I3D_Finetune

TensorFlow code for finetuning I3D model on UCF101.
144 stars 43 forks source link

std::bad_alloc – massive 60GB memory leak? #44

Open shadowlift opened 4 years ago

shadowlift commented 4 years ago

Hi – I'm trying to finetune with two classes. Even running the while loop for a single iteration causes the program to take >60GB of memory, which causes

terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc /var/spool/PBS/mom_priv/jobs/1224593.pbs.SC: line 7: 13498 Aborted (core dumped) python finetune.py ucf101 rgb 1

I'm running python2 with sonnet v1.23, the frame size for my jpeg sequences are 224 and they consist of 200 frames. Any ideas on how to fix?

shadowlift commented 4 years ago

@vra @AlexHu123 @Rhythmblue thanks for your otherwise very helpful work :)