search

janhq / jan

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)
https://jan.ai/
GNU Affero General Public License v3.0
23.74k stars 1.38k forks source link

bug: Stop words input merges spaces with tokens #4088

Closed imtuyethan closed 8 hours ago

imtuyethan commented 4 days ago

Jan version

0.58-734

Describe the Bug

When entering stop words with spaces, input merges spaces into previous token instead of treating them as separate end tokens.

Screenshot 2024-11-22 at 12 03 08 AM

Steps to Reproduce

  1. Open Stop words input
  2. Enter text with spaces between tokens
  3. Observe spaces get merged with previous token