Add vLLM + Llama3-70b example

akash-network / awesome-akash

Awesome List of Akash Deployment Examples

Apache License 2.0

308 stars 223 forks source link

Closed zJuuu closed 5 months ago

zJuuu commented 5 months ago

This PR adds vLLM which allows to easily deploy inference endpoints for many llms on akash network like Llama3.

Dimokus88 commented 5 months ago

I checked the work on the public repository gradientai/Meta-Llama-3-8B-bf16 and it looks great.

Dimokus88 commented 5 months ago

Checked on private repository HF. It's work!

Ready to merge @brewsterdrinkwater !