No EOS when prompt "exit"?

mit-han-lab / TinyChatEngine

TinyChatEngine: On-Device LLM Inference Library

https://mit-han-lab.github.io/TinyChatEngine/

MIT License

756 stars 73 forks source link

Open MoonBlvd opened 5 months ago

MoonBlvd commented 5 months ago

Not sure if this is original llama3 issue or a issue caused by INT4 AWQ. When I type in "exit", it keeps generating without ending the sentence: