search

sigoden / aichat

All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI tools & agents, with access to OpenAI, Claude, Gemini, Ollama, Groq, and more.
Apache License 2.0
4.23k stars 289 forks source link

BUG: when `compress_threshold` is set to 0, ` Session compression is being activated because the current tokens exceed `compress_threshold` is still shown #515

Closed milanglacier closed 5 months ago

milanglacier commented 5 months ago

config:

compress_session: 0

Start with the following session:


me: read the following poem:

...... (copied from a long poem, e.g. https://allpoetry.com/poem/8497263-Atlantis-by-Harold-Hart-Crane)

openai: xxxxxxxx

system: ` Session compression is being activated because the current tokens exceed `compress_threshold` is still shown`

me: repeat the first stanza of the poem, and comment on the first stanza

openai: xxxxxxxx

Side Note

I personally think 1000 is too small as a default value, literally 1000 tokens restricted the conversation within 1-2 replies.

Either turned off by default (the user who uses the expensive LLMs (gpt4, claude3-opus) already takes their own full responsibility for the potential expenses that may yield, just like people who uses free software takes all the responsibility that how potential bugs will influence their apps / computational results)

or at least 8K as the default value.

sigoden commented 5 months ago

I connot reproduce the problem.

Your description is very messy. There is a lot of false information.

  1. The default compress_threshold is 2000, other than 1000.
  2. If compress_threshold < 1000， the compress session will be disabled. The only exception is that session has its own compress_session set separately.

If you can add more information, I will open this issue again.

milanglacier commented 5 months ago

I realized that compress_session is renamed to compress_threshold. Changed compress_session to compress_threshold solves the problem.