search

bentoml / OpenLLM

Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud.
https://bentoml.com
Apache License 2.0
9.7k stars 616 forks source link

feat: Can you support llama3? #961

Closed ICLXL closed 3 months ago

ICLXL commented 4 months ago

Feature request

https://llama.meta.com/llama3/

Motivation

No response

Other

No response

derekhsu commented 4 months ago

Actually you can use llama3 by vllm backend now.

EddyJens commented 3 months ago

Any update? It would be very interesting to have llama3 available

aarnphm commented 3 months ago

llama 3 is already supported.