-
I attempted to run benchmarks for the llama-3-8b-instruct and llama-3.1-8b-instruct models using both CPU and GPU, but the process failed. (I successfully tested the llama2-7b-chatbot model)
I f…
-
Do you have any plans for adding for supporting Llama-3? Any idea how complex this would be, apart from new configs?
-
Bring up Llama 3.1 8b on n150
-
Nvidia now offers llama-3.1 models for example in the same api system. Is it possible to support them? Thanks.
-
https://ai.meta.com/blog/meta-llama-3-1/
-
The trace is collected by modifying the profiling iteration in the vllm benchmark and generating p.export_chrome_trace("trace.json")
at file:
vllm/benchmarks/benchmark_latency.py , func: def r…
-
As llama 2 and llama 3.1 are quite similar in terms of architecture hence, will it be possible to add support for llama 3.1?
-
Does not start with the Llama 3.1 model. Is it possible to make changes to work with Llama 3.1? This is now the model with the most tokens and will potentially be used everywhere.
-
Hi
Please add support for llama-3
Currently the prompt template is not compatible since llama-3 uses different style.
Ref: https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama…
boixu updated
2 weeks ago
-
Please add support for the latest Meta Models
https://ai.meta.com/blog/meta-llama-3-1/