huggingface / hub-docs

Docs of the Hugging Face Hub
http://hf.co/docs/hub
Apache License 2.0
255 stars 222 forks source link

docs: add info about HF pipelines multi-threading in SageMaker inference #1332

Closed kandakji closed 3 weeks ago

kandakji commented 3 weeks ago

Hi,

This PR updates the inference documentation for hugging face on sagemaker. This adds a note about Pipelines lack of multi-threading support, which can represent a CPU bottleneck in sagemaker inference endpoints.