Is there a way to handle the "openai.error.InvalidRequestError" and reduce message length? Can we split the messages into blocks for easier debugging?

[alirezaseifi]

run debugger.py --prompt debug.md --model=gpt-4 ✓ Initialized. View app at https://modal.com/apps/xxxxxxx ✓ Created objects. ├── 🔨 Created generate_response. ├── 🔨 Created mount /Users/a/github/developer/debugger.py └── 🔨 Created mount /Users/a/github/developer/constants.py Traceback (most recent call last): File "/pkg/modal/_container_entrypoint.py", line 330, in handle_input_exception yield File "/pkg/modal/_container_entrypoint.py", line 403, in call_function_sync res = fun(args, kwargs) ^^^^^^^^^^^^^^^^^^^^ File "/root/debugger.py", line 79, in generate_response response = openai.ChatCompletion.create(params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_resources/chat_completion.py", line 25, in create return super().create(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_resources/abstract/engine_apiresource.py", line 153, in create response, , api_key = requestor.request( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 230, in request resp, got_stream = self._interpret_response(result, stream) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 624, in _interpret_response self._interpret_response_line( File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 687, in _interpret_response_line raise self.handle_error_response( openai.error.InvalidRequestError: This model's maximum context length is 8192 tokens. However, your messages resulted in 7334925 tokens. Please reduce the length of the messages. Traceback (most recent call last): File "/pkg/modal/_container_entrypoint.py", line 330, in handle_input_exception yield File "/pkg/modal/_container_entrypoint.py", line 403, in call_function_sync res = fun(args, kwargs) ^^^^^^^^^^^^^^^^^^^^ File "/root/debugger.py", line 79, in generate_response response = openai.ChatCompletion.create(params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_resources/chat_completion.py", line 25, in create return super().create(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_resources/abstract/engine_apiresource.py", line 153, in create response, , api_key = requestor.request( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 230, in request resp, got_stream = self._interpret_response(result, stream) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 624, in _interpret_response self._interpret_response_line( File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 687, in _interpret_response_line raise self.handle_error_response( openai.error.InvalidRequestError: This model's maximum context length is 8192 tokens. However, your messages resulted in 7334925 tokens. Please reduce the length of the messages. Traceback (most recent call last): File "/pkg/modal/_container_entrypoint.py", line 330, in handle_input_exception yield File "/pkg/modal/_container_entrypoint.py", line 403, in call_function_sync res = fun(args, kwargs) ^^^^^^^^^^^^^^^^^^^^ File "/root/debugger.py", line 79, in generate_response response = openai.ChatCompletion.create(params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_resources/chat_completion.py", line 25, in create return super().create(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_resources/abstract/engine_apiresource.py", line 153, in create response, , api_key = requestor.request( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 230, in request resp, got_stream = self._interpret_response(result, stream) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 624, in _interpret_response self._interpret_response_line( File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 687, in _interpret_response_line raise self.handle_error_response( openai.error.InvalidRequestError: This model's maximum context length is 8192 tokens. However, your messages resulted in 7334925 tokens. Please reduce the length of the messages. Traceback (most recent call last): File "/pkg/modal/_container_entrypoint.py", line 330, in handle_input_exception yield File "/pkg/modal/_container_entrypoint.py", line 403, in call_function_sync res = fun(args, kwargs) ^^^^^^^^^^^^^^^^^^^^ File "/root/debugger.py", line 79, in generate_response response = openai.ChatCompletion.create(params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_resources/chat_completion.py", line 25, in create return super().create(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_resources/abstract/engine_apiresource.py", line 153, in create response, , api_key = requestor.request( ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 230, in request resp, got_stream = self._interpret_response(result, stream) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 624, in _interpret_response self._interpret_response_line( File "/usr/local/lib/python3.11/site-packages/openai/api_requestor.py", line 687, in _interpret_response_line raise self.handle_error_response( openai.error.InvalidRequestError: This model's maximum context length is 8192 tokens. However, your messages resulted in 7334925 tokens. Please reduce the length of the messages.

[kcramp858]

I modify debugger.py and add: print(system) print(prompt) print(model)

that way it prints what it is sending to chatgpt; then I look through that and see what I can remove. usually its because there is some cache file or some massive file that isnt necessary needed for your debug..

i also modified that debug program to make a kind of code stripper.. it removes extra white spaces, comments, imports, and logging code.. usually those things are already OK and do not add any context for me.. usually I use this with GPT4 to fit the entire thing in to the token limit. gpt4 doesnt need comments or formatting to understand your code really

minify_no_imports.py:

# minify_no_imports.py
import sys
import tokenize
from io import BytesIO

def minify_python_code(code):
    result, imports = "", ""
    last_end = ""
    g = tokenize.tokenize(BytesIO(code.encode('utf-8')).readline)
    try:
        for tok in g:
            if tok.type == tokenize.NAME and tok.string == 'import':
                imports += tok.line
            elif tok.type in [tokenize.COMMENT, tokenize.ENCODING, tokenize.NL, tokenize.STRING, tokenize.INDENT, tokenize.DEDENT]:
                continue
            elif 'logging.' in tok.line:
                continue
            else:
                if '\n' not in last_end:
                    result += ' '
                result += tok.string
                last_end = tok.string
    except tokenize.TokenError:
        pass
    return imports, result.strip()

if __name__ == "__main__":
    with open(sys.argv[1]) as file:
        code = file.read()
    imports, minified_code = minify_python_code(code)
    print(imports, minified_code, sep="\n")
`

and minify_codebase.py (walks the entire directory and minifies it)

import os
import sys
from minify_no_imports import minify_python_code

def read_file(filepath):
    with open(filepath) as f:
        return f.read()

def write_file(filepath, content):
    with open(filepath, 'w') as f:
        f.write(content)

def walk_directory(directory, skip_list):
    code_contents = {}
    for root, dirs, files in os.walk(directory):
        dirs[:] = [d for d in dirs if d not in skip_list]
        for file in files:
            if file not in skip_list:
                relative_filepath = os.path.relpath(os.path.join(root, file), directory)
                try:
                    code_contents[relative_filepath] = read_file(os.path.join(root, file))
                except Exception:
                    code_contents[relative_filepath] = "error"
    return code_contents

if __name__ == "__main__":
    directory = sys.argv[1] if len(sys.argv) > 1 else os.path.dirname(os.path.abspath(__file__))
    minify_skip_filename = '.minifyskip'
    skip_list = read_file(minify_skip_filename).splitlines()
    codebase = walk_directory(directory, skip_list)
    output_filename = 'minified_codebase.txt'
    imports_filename = 'imports.txt'
    all_imports = ""
    if os.path.exists(output_filename):  # remove existing output file
        os.remove(output_filename)
    for filepath, code in codebase.items():
        if code != "error":
            imports, minified_code = minify_python_code(code)
            all_imports += imports + '\n'
            minified_code_without_imports = minified_code.replace(imports, '')
            with open(output_filename, 'a') as outfile:
                outfile.write(f"{filepath}:\n{minified_code_without_imports}\n")
    write_file(imports_filename, all_imports)

`

fyi im not a programmer, just learning.. seems to work OK. you are trying to send 7334925 tokens, that is.. a lot.. sounds like you are sending some junk files or your codebase is massive

[alirezaseifi]

@kcramp858 thank you so much for your detailed and helpful suggestions!I will definitely try out your code, especially the debug modifications and the Python code minifying script.

You're correct in your observation - I'm dealing with a large project that involves extensive changes on both the backend and frontend, resulting in a lot of code (I also generated the whole code with one prompt with modal main.py). It might indeed be that I'm inadvertently including unnecessary files, so your advice will certainly be valuable in helping me optimize what I'm sending.

Moving forward, I believe a more strategic approach would be to understand the specific areas of the codebase that need modification based on the errors and debug prompts we're encountering. By identifying these relevant files and sending only those and their dependencies to OpenAI, instead of the entire, vast codebase, we could maintain token limits and make the process more efficient and manageable. This is something I'll be looking into more deeply.

By the way, even though you say you're not a programmer, your understanding and approach are really impressive. Keep going on your learning journey!

I'll update the issue once I've made progress with your suggestions. Thanks again for your invaluable input!