Memory Usage

[sdy-ny]

Hi, thank you for this interesting work! I tried it on a short clip and it works great. But when I try to run it on a longer clip with over 3000 frames, it gives me an out of memory error. I already reduced max_ids to 10.

Specifically, the OOM happened when initialing uv_vector in test_tracker.py

track = joblib.load(save_path+'/v1_images'  + '.pickle')   # ~13G
....
pose_emb         = torch.zeros(frame_length, max_ids, 4096)   # 0.5G = 4 * 3000 * 10 * 4096 / 1e9
appe_emb         = torch.zeros(frame_length, max_ids, 4096)   # 0.5G
loca_emb         = torch.zeros(frame_length, max_ids, 99) + 1
uv_vector        = torch.zeros(frame_length, max_ids, 4, 256, 256)   # 31G 

I suppose these vectors are all on cpu, and my machine should have 64G cpu memory. Nonetheless it still errored out. Could you please advise on what is the best way to handle this?

[brjathu]

Hi, are you running it for the demo code?

[sdy-ny]

Yes. I replaced the youtube video with my own video.

[brjathu]

Thanks, just another clarification, are you running this on your local machine right?

[sdy-ny]

Hi, thanks for the quick response! I am running this on a gpu cluster. For this demo code, I requested a single gpu and 64gb memory.

[brjathu]

Great, thanks for your reply. I have added a new file demo_online.py, which is pretty much same as demo.py. But it does not store any detections or HMAR outputs. Therefore, the max memory requirement is now not going to scale with number of frames. Could you please try this and let me know the performance?

[sdy-ny]

Thanks. I tried it and it works great! One possible typo is that there is no render_frame_main_online function, but I replaced it with render_frame_main and it works. Thank you again for the quick response!

[brjathu]

Great!, render_frame_main_online function is inside the make_video.py file.