Closed Extremys closed 11 months ago
Your template has a problem. The stop token of WizardCoder <|endoftext|>
is different from Alpaca </s>
.
thanks for your feedback Chi, I think there is an other because switching from to <|endoftext|> do not resolve the tricky behavior :) any idea?
I try to say Hi!
. It seems to work well.
My suggestion is to use the raw template rather than the template from FastChat. Raw template:
Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
{instruction}
### Response:
Hello Chi, I found the root cause thanks for your feedback it was coming from the worker implementation :)
WizardCoder is really subject to expose roles from the prompt template and continue the discussion like this:
Do you have an idea what could cause this undesirable behavior? How to avoid that? I am using the fastchat repo for inference reusing alpaca prompt template, do you see any bug? https://github.com/lm-sys/FastChat/blob/main/fastchat/conversation.py#L378C26-L378C26
Thanks.