[Bug]

[CodexDive]

Checklist

• [ ] 1. I have searched related issues but cannot get the expected help.
• [ ] 2. The bug has not been fixed in the latest version.

Describe the bug

<|im_start|>user 请背诵一下春江花月夜<|im_end|> <|im_start|>assistant 春江潮水连海平，海上明月共潮生。滟滟随波千万里，何处春江无月明。江流宛转绕芳甸，月照花林皆似霰。空里流霜不觉飞，汀上白沙看不见。江天一色无纤尘，皎皎空中孤月轮。江畔何人初见月？江月何年初照人？人生代代无穷已，江月年年只相似。不知江月待何人，但见长江送流水。白云一片去悠悠，青枫浦上不胜愁。谁家今夜扁舟子？何处相思

double enter to end input >>> 周杰伦是谁

<|im_start|>user 周杰伦是谁<|im_end|> <|im_start|>assistant 周杰伦

Reproduction

lmdeploy "chat", "--backend", "pytorch", "--session-len", "256", "/mnt/self-define/sunning/lmdeploy/internlm2-chat-7b-smooth-quant"

Environment

在chat交互的时候，第一个prompt的结果还相对正常，但是之后再次提交prompt，结果就乱了，很离谱

Error traceback

No response

[CodexDive]

double enter to end input >>> 你是谁？

<|im_start|>user 你是谁？<|im_end|> <|im_start|>assistant 我是一个

[zhyncs]

Hi @CodexDive Does this issue only occur in Smooth Quant or does it also occur in the original model? May you provide a minimal reproducible demo code snippet?

[CodexDive]

不知道是否只有smoothquant，but can you speak chinese? I will try othter quantization method to check whether the chat can work well

[zhyncs]

Sorry but nope. LMDeploy is a project for users around the world. The issues you encounter may also be faced by non-mainland users. Using English for communication is beneficial for other users' use, communication, and participation.

[grimoire]

Unable to reproduce the error. Please provide your environment and how you perform smooth quant.