fix: set correct default value of rope_theta for llama2

vectorch-ai / ScaleLLM

A high-performance inference system for large language models, designed for production environments.

https://docs.vectorch.com/

Apache License 2.0

316 stars 23 forks source link

Closed guocuimi closed 1 month ago

guocuimi commented 1 month ago

set right default value for rope_theta for llama2. also refactor the code a bit to explicately set default values for llama2, llama3 and Yi.

TODOS:

guocuimi commented 1 month ago

This diff would fix the correctness issue for llama2.