search

outlines-dev / outlines

Structured Text Generation
https://outlines-dev.github.io/outlines/
Apache License 2.0
7.05k stars 363 forks source link

Amazon Bedrock Integration #767

Open dgallitelli opened 3 months ago

dgallitelli commented 3 months ago

Before suggesting a new feature, please make sure this hasn't already been suggested by searching through the past issues and the PR tracker.

Presentation of the new feature

Amazon Bedrock is an LLMs as a Service provided by Amazon Web Services. It support multiple Foundation Models, including Amazon Titan, Anthropic Claude 3, Llama 2, and embedding models. To have outlines support Amazon Bedrock, would mean that more than 10000 customers could use it for production.

Where does it fit in Outlines?

New integration, alongside Llama.cpp and OpenAI.

Are you willing to open a PR?

Not a lot of time right now, so unfortunately not yet.

Thanks for contributing!

jeremyallenjacobson commented 2 months ago

See my post in a thread here for why it may not be possible to support various models on Bedrock.