search

tloen / alpaca-lora

Instruct-tune LLaMA on consumer hardware
Apache License 2.0
18.55k stars 2.21k forks source link

Got much different example outputs #394

Open tigerchen52 opened 1 year ago

tigerchen52 commented 1 year ago

Hi authors,

Thank you for publishing the codes! Nice work!

I have run this demo successfully, but got very different outputs when I was playing with the example instructions:

Instruction: Tell me about alpacas. Response: Alpacas are members of the camelid family and are native to the Andes Mountains in South America. They are known for their soft and luxurious fleece, which can be spun into yarn. Alpacas can be found in a variety of colors, including white, grey, brown, black, and fawn. Alpacas are herd animals and can live up to 25 years in captivity.

Instruction: Tell me about the president of Mexico in 2019. Response: President of Mexico in 2019 is Andres Manuel Lopez Obrador. He was sworn in on December 1st, 2018. He is the first left-wing president of Mexico since 1988. He is known for his anti-corruption and anti-neoliberal policies.

Instruction: Tell me about the king of France in 2019. Response: The King of France in 2019 is Emmanuel Macron, who was elected President of France in May 2017. He is the youngest President of France since Napoleon Bonaparte in 1799, and the first President of France to be born in the 1970s. He is also the first President of France to be born after World War II.

Instruction: List all Canadian provinces in alphabetical order. Response: Alberta, British Columbia, Manitoba, New Brunswick, Newfoundland and Labrador, Nova Scotia, Ontario, Prince Edward Island, Quebec, Saskatchewan.

I'd like to do some research work based on this repo, but I'm not sure I have installed everything correctly because the example outputs are different from yours. Is this very normal to happen or I made something wrong? Could you give me some suggestions? Thanks in advance.

pandermatt commented 1 year ago

Hi @tigerchen52

Can you check which temperature you used?

The temperature parameter controls the randomness of the output, with higher values leading to more diverse responses, and lower values making the output more deterministic.

If you want to get more consistent results, you can indeed lower the temperature parameter. Setting it to 0 might make the output too deterministic and repetitive, so it's recommended to use a small value instead (e.g., 0.1 or 0.2). This should provide more consistent results while still retaining some level of diversity in the generated text.

I'm not the author, but maybe this is helpful to you!