search

huggingface / optimum-tpu

Google TPU optimizations for transformers models
Apache License 2.0
76 stars 19 forks source link

Lower TGI IE batch size #71

Closed tengomucho closed 4 months ago

tengomucho commented 4 months ago

What does this PR do?

This lowers the default batch size to 2 to avoid memory issues with some models, and increments version.

HuggingFaceDocBuilderDev commented 4 months ago

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.