Automatic batch size using accelerate

VikParuchuri / marker

Convert PDF to markdown quickly with high accuracy

https://www.datalab.to

GNU General Public License v3.0

16.82k stars 955 forks source link

Automatic batch size using accelerate #115

Closed juletx closed 4 months ago

juletx commented 5 months ago

It would be great to add an option to select batch size automatically using accelerate find_executable_batch_size function: https://huggingface.co/docs/accelerate/main/en/package_reference/utilities#accelerate.find_executable_batch_size

VikParuchuri commented 4 months ago

I'm not planning to include this in the core marker (I don't want to bloat the library), but it would be easy to write an external utility that does this