Script stops running after a while with big file sizes (~80-100MB).

[joeamroo]

Hi! So I'm running a script that would take 40ish files using 16 workers. Problem is it stops working after a while

It almost always stops after the last few files get their reading order done.

[joeamroo]

@VikParuchuri Do you know why that might be happening? left it running overnight and it got stuck for some reason, I have an A100 with 5.5GB VRAM_PER_TASK.

[joeamroo]

It gets stuck after finishing Detecting bboxes and the processes just rest at those memory sizes.

[joeamroo]

205 Might be related.