OpenCoderPlus outputs will not stop

[HermitSun]

I tried to launch OpenCoderPlus with the latest code of this repo and vLLM:

python -m ochat.serving.openai_api_server --model-type opencoder --model openchat/opencoderplus

It can work, but the outputs will never stop util hitting the max_tokens limit, even if I pass the stop parameter:

requests.post(
  "http://localhost:18888/v1/chat/completions",
  json={
    "model": "opencoder",
    "messages": [{"rule": "user", "content": "Write a bubble sort."}],
    "stop": ["<|end_of_turn|>"]
  }
) 

I refered to OpenCoderPlus's training data, it seems that this model is training on data with the <|end_of_turn|> character.

So does anyone know how to stop this model's outputs? Any help will be appreciated.

[imoneoi]

It's because the new vLLM doesn't decode special tokens. We've fixed it by replacing the stop token, can you try again?

[HermitSun]

Thank you for your reply. Could you tell me what's the stop token now?

Or furthermore, is there any example of calling OpenCoderPlus correctly?

Thanks in advance.