search

bramses / chatgpt-md

A (nearly) seamless integration of ChatGPT into Obsidian.
MIT License
854 stars 64 forks source link

Can not continue conversion #17

Closed nkmechanics closed 1 year ago

nkmechanics commented 1 year ago

Hi, I updated to the most recent version today.

I tried a few examples, it works perfectly at the beginning, but whenever I'd like to continue the conversion it will show error like this:

[ChatGPT MDI Error :: This model's maximum context length is 4097 tokens. However, you requested 4621 tokens (621 in the messages, 4000 in the completion). Please reduce the length of the messages or completion. issue calling OpenAl API, see console for more details

Am I doing it wrong or I can only have 4097 tokens for the whole note?

nkmechanics commented 1 year ago

It seems I set max_tokens too high (4000), I changed to 512 and now it's working... Don't know why though...

bramses commented 1 year ago

Hi @nkmechanics, I believe the API automatically rejects anything before it calculates that would go above 4096 tokens. So in your case 621 in the messages + 4000 in the completion would overload the system so it pre rejected it. Does that make sense?

nkmechanics commented 1 year ago

ok, so since I set max_tokens to 4000, it will always use this value to pre-compute the total tokens? It does make sense to think this way...

I was setting it high in the hope to get more responds without the cutoff. Guess I need to play with the value to get an optimized results...

Thanks for the explanation!

nkmechanics commented 1 year ago

Another question: Is the limit of 4096 token applies to the whole conversions (a.k.a., the whole note)? Not a single response (one question and one answer)?

bramses commented 1 year ago

The whole conversation, yes. There are tricks you can do with summarizing messages or building a "sliding window" but your best bet w ChatGPT MD is just to create a new chat and go from there

nkmechanics commented 1 year ago

Thanks for the confirmation!