Adding support for running HuggingFace models on AWS Inferentia

weaviate / t2v-transformers-models

This is the repo for the container that holds the models for the text2vec-transformers module

BSD 3-Clause "New" or "Revised" License

40 stars 27 forks source link

Adding support for running HuggingFace models on AWS Inferentia #28

Open zoltan-fedor opened 2 years ago

zoltan-fedor commented 2 years ago

To achieve much faster inference.

See https://huggingface.co/blog/bert-inferentia-sagemaker#1-convert-your-hugging-face-transformer-to-aws-neuron

weaviate-git-bot commented 2 years ago

To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge.

beep boop - the SeMI bot 👋🤖

zoltan-fedor commented 2 years ago

I agree with the CLA.

bobvanluijt commented 2 years ago

@zoltan-fedor – apologies for the late response and thanks for your PR! We are on it :-)