search

liltom-eth / llama2-webui

Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.
MIT License
1.96k stars 202 forks source link

Why can llama understand Chinese so much and can't reply directly in Chinese? #17

Open XksA-me opened 1 year ago

XksA-me commented 1 year ago

Why can't llama understand Chinese so much and can't reply directly in Chinese?

I tested Llama-2-7b-chat-hf again today.

Test using GPU platform: matpool.com Memory usage: Open 8BIT occupies 8G+, GPU utilization: 13-20% If 8BIT is not enabled, it takes up 14G+, GPU utilization: 55-70%

image image image
liltom-eth commented 1 year ago

I believe this happened because of the system prompt. this is added to your chat text sending to llm. Try delete system prompt in UI or use Chinese system prompt. The results might be better. Screenshot 2023-07-27 at 1 11 24 AM

Lllama2 I believe not well supported on Chinese now since Chinese have more tokens.

I list a feature to support some Chinese llama2 models. Haven't got a chance to test it.

XksA-me commented 1 year ago

thanks for the reply. Useful, but limited.

image image

Hope I can contribute for feature.

liltom-eth commented 1 year ago

@XksA-me do you want to contribute to this feature? Might need take a look at this repo Chinese-Llama-2-7b.

liltom-eth commented 1 year ago

@XksA-me welcome contributing your benchmark performance here.