Bug with context window

clairefro / obsidian-chat-cbt-plugin

AI-powered journaling plugin for your Obsidian notes, inspired by cognitive behavioral therapy

https://obsidian.md/plugins?search=chatcbt

MIT License

189 stars 11 forks source link

Bug with context window #7

Open IbragimovDP opened 11 months ago

IbragimovDP commented 11 months ago

When I have a long conversation I start to receive error 400 probably because of limits of context window. I think so because when I change model from 3.5 with 4k window to 4 with 128k window I can continue my conversation.

clairefro commented 9 months ago

Thinking out loud: some options

select a model that has a longer context window if you need a longer conversation
break convo into two or more notes
add a feature to summarize (compress) the convo when it is approaching context window

IbragimovDP commented 9 months ago

Just idea: Add tokens_value property for message object and send to gpt all last messages that fitted to context window. But in that case we'll have some glitches, when CBT forgets chat's beginning.