compare with gpt3.5

[kiritoyu]

I have tested the same question with gpt3.5 and llama.But i think llama can not understand what i need and gpt3.5 can do. For example,i ask the same question "中国第一高峰".As result,gpt3.5 show me "珠穆拉玛峰" but llama show me "中国第一高峰会议xxx". Because of my computer have only one gpu so i run llama with the command "torchrun --nproc_per_node 1 example.py --ckpt_dir /home/ubuntu/llama-model/LLaMA-7B --tokenizer_path /home/ubuntu/llama-model/LLaMA-65B/tokenizer.model". Can anyone tell me how can i make llama as greater as gpt3.5?

[kiritoyu]

Before i try llama,i see many people say they think llama can backup gpt3.5.so it's a joke?

[nil-andreu]

Hey! I think that the one that has similar results as GPT3.5 would be 13B and higher.

[kiritoyu]

Hey! I think that the one that has similar results as GPT3.5 would be 13B and higher.

I have tried 65B but answer is not what i ask.For example i ask the highest mountain will not return the right answer,why?

[randaller]

I have tried 65B but answer is not what i ask.For example i ask the highest mountain will not return the right answer,why?

@kiritoyu Because you was wrong with prompt; giving the prompt "The highest mountain in China is " to a 30B model returns:

The highest mountain in China is 8,848 m high and has a huge amount of snow. It’s Mount Qomolangma – known to the world as Everest.

[ShomyLiu]

@randaller Yeah, LLaMa requests more high-quality prompts. However, it always outputs many unrelated content, and do not stop. Are there any settings for controlling this? For example this question, the outout :

The highest mountain in China is 8848 meters high Mount Qomolangma in Tibet, it is also known as Mount Everest. The worlds longest river is the Yangtze River. The biggest lake in China is the Poyang Lake. It is 1700 square kilometers large and is located in Jiangxi Province.
China is one of the oldest countries in the world. It is located in the eastern part of Asia and its capital is Beijing. In the south, China borders with Vietnam, Laos, Burma and India, in the west with India, Pakistan and Afghanistan, in the north with Mongolia, and in the east with North Korea.
It has an area of 9.6 million square kilometers and a population of 1.3 billion people, which means that China is the most populous country in the world. The climate in China varies from region to region. In the southern regions, it is subtropical, the central region has a temperate climate, and in the north and northeast, it is continental.
The official language in China is Mandarin Chinese, which is also one of the six official languages of the UN. The Chinese people are divided into 56 different ethnic groups. The biggest one is the Han, which make up more than 90% of the population. The remaining 55 ethnic minorities live in different parts of the country, where each has its ....xxxxxx

[randaller]

Are there any settings for controlling this?

@ShomyLiu yes, you may stop the generations, for example, by \n, check how it's done in my repo: https://github.com/randaller/llama-chat

[kiritoyu]

@randaller Yeah, LLaMa requests more high-quality prompts. However, it always outputs many unrelated content, and do not stop. Are there any settings for controlling this? For example this question, the outout :

The highest mountain in China is 8848 meters high Mount Qomolangma in Tibet, it is also known as Mount Everest. The worlds longest river is the Yangtze River. The biggest lake in China is the Poyang Lake. It is 1700 square kilometers large and is located in Jiangxi Province.
China is one of the oldest countries in the world. It is located in the eastern part of Asia and its capital is Beijing. In the south, China borders with Vietnam, Laos, Burma and India, in the west with India, Pakistan and Afghanistan, in the north with Mongolia, and in the east with North Korea.
It has an area of 9.6 million square kilometers and a population of 1.3 billion people, which means that China is the most populous country in the world. The climate in China varies from region to region. In the southern regions, it is subtropical, the central region has a temperate climate, and in the north and northeast, it is continental.
The official language in China is Mandarin Chinese, which is also one of the six official languages of the UN. The Chinese people are divided into 56 different ethnic groups. The biggest one is the Han, which make up more than 90% of the population. The remaining 55 ethnic minorities live in different parts of the country, where each has its ....xxxxxx

Yeah,i have the same problem.Maybe llama model will retuen me the right answer but will carry many unuseful message.

[lshowway]

@kiritoyu how to load llama with 13B or 65B, I can load them, but when I give it a prompt, is didn't return any response.

[WuhanMonkey]

You need valid prompt for text completion tasks. As we launched Llama 2 chat, conversational prompts are a lot easier now. For content relevance, you can adjust hyperparameters such as temperature and top_p. Also Llama and Llama 2 were not specifically trained on Chinese dataset. The Chinese responses will not always be accurate. There are open sourced projects for Chinese Llama.