search

ray-project / ray-llm

RayLLM - LLMs on Ray
https://aviary.anyscale.com
Apache License 2.0
1.22k stars 89 forks source link

Missing Llama2 policy #49

Closed NikitaSemenovAiforia closed 10 months ago

NikitaSemenovAiforia commented 1 year ago

I got an error: ValueError: Model meta-llama/Llama-2-7b-chat-hf cannot automatically infer max_batch_total_tokens. Make sure to set engine_config.scheduler.policy.max_batch_total_tokens in the model configuration yaml.

akshay-anyscale commented 10 months ago

this should be fixed now, please retry