Memory Issue on all projects

VikParuchuri / surya

OCR, layout analysis, reading order, table recognition in 90+ languages

https://www.datalab.to

GNU General Public License v3.0

14.42k stars 908 forks source link

Memory Issue on all projects #254

Open moshetanzer opened 3 days ago

moshetanzer commented 3 days ago

Hi,

I have been getting this issue on all projects whether 1 page or 20 pages also on marker pdf

 attn_output = torch.nn.functional.scaled_dot_product_attention(
RuntimeError: MPS backend out of memory (MPS allocated: 7.23 GB, other allocations: 1.54 GB, max allowed: 9.07 GB). Tried to allocate 896.00 MB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).

moshetanzer commented 3 days ago

Update it is only when doing multiple files together. Pretty sure batch 512 is too big. Just very difficult to judge how many - since 32 takes 15 min to do a 10 page doc. Would be great to get guidance. Thanks again.