search

jupediaz / chatgpt-prompt-splitter

ChatGPT PROMPTs Splitter. Tool for safely process chunks of up to 15,000 characters per request
https://chatgpt-prompt-splitter.jjdiaz.dev
MIT License
493 stars 82 forks source link

If the the context token limit is 8k, then wouldn't gpt start forgetting the initial chunks? #16

Open dhyanKaro opened 9 months ago

dhyanKaro commented 9 months ago

15000 chars ~= 5200 tokens Well after a few chunks, the initial text wouldn't be context anymore, right?

adamerose commented 9 months ago

Yes you're right, I thought the same thing and came here looking for an explanation. It seems like rather than completely forgetting the first chunk, it tries to compress it. The more chunks you include, the more detail is lost.

My guess is that when your conversation history reaches the GPT memory limit, ChatGPT gets GPT to summarize the conversation so far and includes that as the history for subsequent messages rather than the full text.

For example I took the full text of this paper and used chatgpt-prompt-splitter to break it into 3 chunks. ChatGPT then gave me a summary of the paper, but when I asked "Give me the full text of the Abstract" it failed to do so. If I only paste the first chunk and ask the same question, it doesn't fail.

https://chat.openai.com/share/69457db1-b3dc-4224-a0ba-538c14fa4292 https://chat.openai.com/share/bd022ee2-5f9e-4d25-8958-63a1ec667184