search

rui-ye / FedLLM-Bench

Other
57 stars 4 forks source link

Fed-Aya prompt template #4

Closed royson closed 2 weeks ago

royson commented 2 weeks ago

Hi, I enjoyed reading the paper, great work!

I have some perhaps naive questions regarding Fed-Aya's prompt template. Is the prompt template always in English for all the different languages (https://github.com/rui-ye/FedLLM-Bench/blob/main/training_scripts/FedAya/aya_all.sh#L40)? If yes, why not use the template of the corresponding languages? Thanks!

rui-ye commented 2 weeks ago

Hi, thanks for your recognition! Yes, the prompt template is always in English for all the different languages. The reason is that in the deployment of a multilingual LLM, the LLM provider cannot not know which language will the user use in advance. Therefore, using a unified template is better.

royson commented 2 weeks ago

Thanks for your quick response! I have some follow-up questions: if the LLM (llama-7b in your case) is tuned on multilingual prompt templates, I supposed it should be able to handle them during deployment? Or perhaps the main limitation is the llama tokenizer, which mainly does well for some languages like English, German, French but not others like Greek and Chinese?

rui-ye commented 2 weeks ago

I think the main point is that the LLM provider offers model service for users all over the world, but cannot know which language the user will use. Therefore, the LLM provider cannot choose the template accordingly (unless something like a language classifier is introduced).

Yeah, I think the tokenizer is also related to such choice.

royson commented 2 weeks ago

Ah, I got it now. Thanks for the explanation! 👍