search

Omegastick / SillyTavern-CacheChunker

Extension for improving cache utilization in Silly Tavern
7 stars 2 forks source link

Upper limit #2

Open GHBigD opened 6 months ago

GHBigD commented 6 months ago

Is there an upper limit to the max context the script can handle? If I set it to 32,000 it seems to work fine. I bumped it to 60,000 to handle a 65k context model, and it trimmed down to about 12,000 for some reason. Put it back to 32,000, and it picked up the extra chat history and started using it right away. Bump it back to 60,000 after I get a reply at 32,000, and it cuts me right back down to around 12,000 I have not played with numbers between 32,000 and 60,000 to see where the cutoff is.

Vhallo commented 4 months ago

Annoying bug indeed, happens because the starting point is reversed. But I fixed that.

GHBigD commented 4 months ago

Annoying bug indeed, happens because the starting point is reversed. But I fixed that.

Thanks! I just came to see if any progress had been made because I encountered it this week. Happy to see you worked on it! I am pulling it down now.