search

Vali-98 / ChatterUI

Simple frontend for LLMs built in react-native.
GNU Affero General Public License v3.0
591 stars 31 forks source link

Chat message was cut off #91

Closed GeminiX369 closed 3 weeks ago

GeminiX369 commented 1 month ago

When using the Chat Completions API, the messages sent to the server during the chat process will only intercept the latest messages based on the size of Generated Tokens. This will cause the server to only see short-term content and lose the complete context, seriously affecting normal use. I hope the developers can fix this problem, or at least provide a switch that allows users to choose whether to only intercept and send the most recent messages.

Vali-98 commented 1 month ago

The app itself cuts messages based on the API's recommended context limits, though it is possible to completely bypass this and send everything, it wouldnt be recommended to do so. That said, it can be added as a user setting.

GeminiX369 commented 1 month ago

The app itself cuts messages based on the API's recommended context limits, though it is possible to completely bypass this and send everything, it wouldnt be recommended to do so. That said, it can be added as a user setting.

Thanks for the reply. Yes I know this, in fact most gpt clients intercept historical messages sent. However, the context intercepted by ChatterUI is too short. Even with the maximum number of tokens, there are only a few messages. I think the context of at least a dozen messages is considered normal.

Vali-98 commented 1 month ago

the context intercepted by ChatterUI is too short

Could you provide an example of this happening?

GeminiX369 commented 1 month ago

the context intercepted by ChatterUI is too short

Could you provide an example of this happening?

I wanted to export the logs but failed because the software did not have permission to access the Download folder. You can try letting it tell you ghost stories constantly, and then check the logs and you will find that only the system prompt and the last few messages have been sent.

Vali-98 commented 3 weeks ago

I spent today checking back on how the API builders function, apparently generated length controlled the actual context length which is a major inaccuracy.

I will try to get a build out to fix this.

GeminiX369 commented 3 weeks ago

I spent today checking back on how the API builders function, apparently generated length controlled the actual context length which is a major inaccuracy.

I will try to get a build out to fix this.

Yes, thanks for your work.

Vali-98 commented 3 weeks ago

Should be fixed in: https://github.com/Vali-98/ChatterUI/releases/tag/v0.8.1

Feel free to reopen the issue if this still occurs.