Creative Commons Attribution Share Alike 4.0 International
242
stars
52
forks
source link
When should we set `example_prefix` to be True? And what is the difference between put ICL examples into system prompt versus multi-turn user-assistant chat? #49
I understand that example_prefix is used during ICL so that we can put examples into the system prompt, especially when we are using GPT-4. However, I have several questions regarding this feature:
What is the difference between putting ICL examples into system prompts versus putting them into multi-turn conversations between user and assistant after system prompt?
Is it recommend only for GPT-4 or both GPT-4 and GPT-3.5? I found there are some discussion regarding the difference between GPT-4 and GPT-3.5 when they are taking system prompt with different name attributes, what is the suggestion from the authors regarding this issue?
If my understanding is correct, when using open source models such as Llama-2, we shouldn't set example_prefix to be true as this will cause some issues with the prompt. To be more concrete, when calling the HFChat class in the src/dt/chat.py at line 395, if we are using example_prefix, we will construct multiple system prompts (2 for each ICL example, with name of example_user and example_assistant). However, at line 395, the implementation seems to overlap previous system prompt with the latest system prompt (i.e., conv.system) rather than concatenate multiple system prompts together, which will cause issue when loading multiple system prompts.
Please help me verify my understandings here. Thanks!
I understand that
example_prefix
is used during ICL so that we can put examples into the system prompt, especially when we are using GPT-4. However, I have several questions regarding this feature:user
andassistant
after system prompt?name
attributes, what is the suggestion from the authors regarding this issue?example_prefix
to be true as this will cause some issues with the prompt. To be more concrete, when calling theHFChat
class in thesrc/dt/chat.py
at line 395, if we are usingexample_prefix
, we will construct multiple system prompts (2 for each ICL example, with name ofexample_user
andexample_assistant
). However, at line 395, the implementation seems to overlap previous system prompt with the latest system prompt (i.e.,conv.system
) rather than concatenate multiple system prompts together, which will cause issue when loading multiple system prompts.Please help me verify my understandings here. Thanks!