Weird output from LLaMA-3.1-70B-Instruct

meta-llama / llama-models

Utilities intended for use with Llama models.

Other

3.81k stars 685 forks source link

Weird output from LLaMA-3.1-70B-Instruct #124

Open Ignoramus0817 opened 3 weeks ago

Ignoramus0817 commented 3 weeks ago

We find that LLaMA-3.1-70B-Instruct outputs duplicated brackets at a high frequency. For example,

Since you didn't provide the full error message, I couldn `[ [ [ [ [ [ [ [ [ [ [ [ [ [ []]]]]]]]]]]]]]]]]]]]. If you still encounter issues, please provide the full error message......

We use vllm-0.5.3-post1 as the inference background. Do anyone has ideas on this behavior? Thanks a lot.

wukaixingxp commented 3 weeks ago

Hi! May I know your prompt to test this? Can you double check if the 70B model is the latest? The chat template has been changed in this PR, please make sure 70B model's chat template is the latest.