EmpathicSage
commented
1 year ago This has been reported numerous times across multiple issues, and the core contributors are already aware of and working on it.
That said...
Through trial and error, and as previously mentioned, I also believe the optimal solution lies in segmenting the requests into "chunks," akin to the method employed by the Superpower ChatGPT plugin. I will explain.
With ChatGPT 3.5, a token budget of 4097 is allocated, which can be utilized for either input, output or a combination of both.
The issue arises when Auto-GPT transmits a considerable volume of data, consuming all the allocated tokens, and leaving none for the response. Alternatively, truncating the data sent to ChatGPT results in errors during the response creation and handling.
Therefore, the proposed fix involves identifying the total token count using a tokenizer on the input text, dividing the request into segments or 'chunks,' appending the pre and post-sections, and progressively submitting them until the quota is exhausted. The submission would be divided into 'X' parts, where 'X' is a factor of (4000 - pre/post section token length).
For instance, here's how Superpower ChatGPT effectively implements this strategy:
Act like a document/text loader until you load and remember the content of the next text/s or document/s.
There might be multiple files, each file is marked by name in the format ### DOCUMENT NAME.
I will send them to you in chunks. Each chunk starts will be noted as [START CHUNK x/TOTAL], and the end of this chunk will be noted as [END CHUNK x/TOTAL], where x is the number of current chunks, and TOTAL is the number of all chunks I will send you.
I will split the message in chunks, and send them to you one by one. For each message follow the instructions at the end of the message.
Let's begin:
[START CHUNK 1/2]
... THE CHUNK CONTENT GOES HERE ...
[END CHUNK 1/2]
Reply with OK: [CHUNK x/TOTAL]
Don't reply with anything else!Superpower ChatGPT on the Google Chrome webstore: https://chrome.google.com/webstore/detail/superpower-chatgpt/amhmeenmapldpjdedekalnfifgnpfnkc
See also: https://github.com/saeedezzati/superpower-chatgpt See also: https://medium.com/@josediazmoreno/break-the-limits-send-large-text-blocks-to-chatgpt-with-ease-6824b86d3270
logichammer
Monty1122
Krim10000
ecarradal
JmMndz
katmai
EncryptShawn
koroteeww
chrisduk112
kinance
⚠️ Search for existing issues first ⚠️
Which Operating System are you using?
Docker
Which version of Auto-GPT are you using?
Master (branch)
GPT-3 or GPT-4?
GPT-3.5
Steps to reproduce 🕹
listing the auto_gpt_workspace folder errors out. maybe this is an erroneous bug, not really sure, but why is it calling openai when it's merely listing the files in the folder?
openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 4819 tokens. Please reduce the length of the messages.
Current behavior 😯
listing the folder contents errors out and kills the program if there's too many files in there.
Expected behavior 🤔
not ... error out :D
Your prompt 📝
Your Logs 📒