search

mlabonne / llm-course

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.
https://mlabonne.github.io/blog/
Apache License 2.0
39.18k stars 4.14k forks source link

Prompt is getting repeated in response #2

Closed physaikat closed 1 year ago

physaikat commented 1 year ago

I tried to retrain Llama-2 model. I just followed the steps you have mentioned. But when I am generating the text with the following code snippet - 

prompt = "What is a large language model?"
pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=200)
result = pipe(f"[INST] {prompt} [/INST]")
print(result[0]['generated_text'])

I am getting a weird response as below - 

[INST] What is a large language model? [/INST]
[INST] What is a large language model? [/INST]
[INST] What is a large language model? [INST]
[INST] What is a large language model? [INST]
[INST] What is a large language model? [INST] [INST] What is a large language model? [/INST]
[INST] What is a large language model? [INST] [INST] What is a large language model? [/INST] [INST] What is a large language model? [INST] [INST] What is a large language model? [/INST] [INST] What is a large language model? [INST] [INST] What is a large language model? [/INST] [INST] What is a large language model? [INST] [INST] What is a large language model? [/INST] [INST] What is a

What could be the issue?

mlabonne commented 1 year ago

That's weird, this is the first time I've seen this behavior with this model. Here's what I would recommend:

  1. Using a non-chat Llama 2 model
  2. Train it on the full Guanaco dataset (you can use the original dataset timdettmers/openassistant-guanaco)