search

bentoml / OpenLLM

Run any open-source LLMs, such as Llama, Mistral, as OpenAI compatible API endpoint in the cloud.
https://bentoml.com
Apache License 2.0
10.11k stars 640 forks source link

feat: Can you support llama3? #961

Closed ICLXL closed 6 months ago

ICLXL commented 7 months ago

Feature request

https://llama.meta.com/llama3/

Motivation

No response

Other

No response

derekhsu commented 7 months ago

Actually you can use llama3 by vllm backend now.

EddyJens commented 6 months ago

Any update? It would be very interesting to have llama3 available

aarnphm commented 6 months ago

llama 3 is already supported.