Open exo-pla-net opened 1 year ago
I can confirm that. Using GPT-4 and fed it 22 sections of a document I have, it just keeps answering within the context of the last two. When I tried to be smart and confront it 😂 this is what it said:
Unfortunately, I don't have access to the earlier sections of the document,
so I'm unable to verify if there are any missing procedures.
If you have any specific questions or need further details about
the sections I've summarized, please let me know!
Though you can indeed split up a long text into chunks, and you then feed them all into a ChatGPT chat, this is irrelevant.
The maximum input length in a ChatGPT prompt is equivalent to the maximum context size for ChatGPT. Thus, ChatGPT will "forget" all the previous snippets, and it will only know about the final snippet.
Try it: use this script to break apart a long text. Then quiz it on a detail only present in the first snippet. ChatGPT will have no idea about it, since it's outside of its current context, which is ONLY THE FINAL SNIPPET. (And maybe a bit of the second-to-last snippet, if the final snippet is less than the maximum input length.)
This problem is impossible to solve in a fully satisfactory way. I'd love to have a longer context, too, but this is unfortunately not solvable by breaking apart a long text. The best you can do is have ChatGPT iteratively summarize previous snippets, then feed the summary into the next snippet. You can programmatically do this, and it might work for your purposes.
As for this project, it should have a disclaimer at the top that this script is useless / just for fun, because it's currently just wasting people's time. If you want to salvage this project into something useful, then here's what it could look like:
The above is a script that would indeed be useful. But the current one is a waste of time and should ethically have a disclaimer at the top.