InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 7575 tokens. Please reduce the length of the messages.

[kcramp858]

While following the workflow shown in the YouTube video, I run into this error.

I had run a prompt and gotten an error. I tried running debugger on the traceback and it errors out.

Command is here: modal run debugger.py --prompt "codespace ➜ /workspaces/developer (main) $ /home/codespace/.python/current/bin/python3 /workspaces/developer/generated/main.py Enter the path to the CSV file: data.csv Traceback (most recent call last): File "/workspaces/developer/generated/main.py", line 16, in <module> main() File "/workspaces/developer/generated/main.py", line 10, in main column_names, data_rows = csv_parser.parse_csv(csv_file_path) TypeError: parse_csv() missing 1 required positional argument: 'column_indices'"

Error is here:

✓ Initialized. View app at https://modal.com/apps/ap-P7vk4JqfCQJSOKa1GkPKfC
✓ Created objects.
├── 🔨 Created generate_response.
└── 🔨 Created mount /workspaces/developer/debugger.py
Traceback (most recent call last):
  File "/pkg/modal/_container_entrypoint.py", line 330, in handle_input_exception
    yield
  File "/pkg/modal/_container_entrypoint.py", line 403, in call_function_sync
    res = fun(*args, **kwargs)
  File "/root/debugger.py", line 80, in generate_response
    response = openai.ChatCompletion.create(**params)
  File "/usr/local/lib/python3.10/site-packages/openai/api_resources/chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 230, in request
    resp, got_stream = self._interpret_response(result, stream)
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 624, in _interpret_response
    self._interpret_response_line(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 687, in _interpret_response_line
    raise self.handle_error_response(
openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 7575 tokens. Please reduce the length of the messages.
Traceback (most recent call last):
  File "/pkg/modal/_container_entrypoint.py", line 330, in handle_input_exception
    yield
  File "/pkg/modal/_container_entrypoint.py", line 403, in call_function_sync
    res = fun(*args, **kwargs)
  File "/root/debugger.py", line 80, in generate_response
    response = openai.ChatCompletion.create(**params)
  File "/usr/local/lib/python3.10/site-packages/openai/api_resources/chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 230, in request
    resp, got_stream = self._interpret_response(result, stream)
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 624, in _interpret_response
    self._interpret_response_line(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 687, in _interpret_response_line
    raise self.handle_error_response(
openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 7575 tokens. Please reduce the length of the messages.
Traceback (most recent call last):
  File "/pkg/modal/_container_entrypoint.py", line 330, in handle_input_exception
    yield
  File "/pkg/modal/_container_entrypoint.py", line 403, in call_function_sync
    res = fun(*args, **kwargs)
  File "/root/debugger.py", line 80, in generate_response
    response = openai.ChatCompletion.create(**params)
  File "/usr/local/lib/python3.10/site-packages/openai/api_resources/chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 230, in request
    resp, got_stream = self._interpret_response(result, stream)
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 624, in _interpret_response
    self._interpret_response_line(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 687, in _interpret_response_line
    raise self.handle_error_response(
openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 7575 tokens. Please reduce the length of the messages.
Traceback (most recent call last):
  File "/pkg/modal/_container_entrypoint.py", line 330, in handle_input_exception
    yield
  File "/pkg/modal/_container_entrypoint.py", line 403, in call_function_sync
    res = fun(*args, **kwargs)
  File "/root/debugger.py", line 80, in generate_response
    response = openai.ChatCompletion.create(**params)
  File "/usr/local/lib/python3.10/site-packages/openai/api_resources/chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 230, in request
    resp, got_stream = self._interpret_response(result, stream)
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 624, in _interpret_response
    self._interpret_response_line(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 687, in _interpret_response_line
    raise self.handle_error_response(
openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 7575 tokens. Please reduce the length of the messages.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/codespace/.python/current/bin/modal:8 in <module>                                          │
│                                                                                                  │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(main())                                                                         │
│   9                                                                                              │
│                                                                                                  │
│ /usr/local/python/3.10.4/lib/python3.10/site-packages/modal/__main__.py:6 in main                │
│                                                                                                  │
│    5 def main():                                                                                 │
│ ❱  6 │   entrypoint_cli()                                                                        │
│    7                                                                                             │
│                                                                                                  │
│ /usr/local/python/3.10.4/lib/python3.10/site-packages/click/core.py:1130 in __call__             │
│                                                                                                  │
│   1129 │   │   """Alias for :meth:`main`."""                                                     │
│ ❱ 1130 │   │   return self.main(*args, **kwargs)                                                 │
│   1131                                                                                           │
│                                                                                                  │
│ /usr/local/python/3.10.4/lib/python3.10/site-packages/typer/core.py:778 in main                  │
│                                                                                                  │
│   777 │   ) -> Any:                                                                              │
│ ❱ 778 │   │   return _main(                                                                      │
│   779 │   │   │   self,                                                                          │
│                                                                                                  │
│ /usr/local/python/3.10.4/lib/python3.10/site-packages/typer/core.py:216 in _main                 │
│                                                                                                  │
│   215 │   │   │   with self.make_context(prog_name, args, **extra) as ctx:                       │
│ ❱ 216 │   │   │   │   rv = self.invoke(ctx)                                                      │
│   217 │   │   │   │   if not standalone_mode:                                                    │
│                                                                                                  │
│ /usr/local/python/3.10.4/lib/python3.10/site-packages/click/core.py:1657 in invoke               │
│                                                                                                  │
│   1656 │   │   │   │   with sub_ctx:                                                             │
│ ❱ 1657 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))               │
│   1658                                                                                           │
│                                                                                                  │
│ /usr/local/python/3.10.4/lib/python3.10/site-packages/click/core.py:1657 in invoke               │
│                                                                                                  │
│   1656 │   │   │   │   with sub_ctx:                                                             │
│ ❱ 1657 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))               │
│   1658                                                                                           │
│                                                                                                  │
│ /usr/local/python/3.10.4/lib/python3.10/site-packages/click/core.py:1404 in invoke               │
│                                                                                                  │
│   1403 │   │   if self.callback is not None:                                                     │
│ ❱ 1404 │   │   │   return ctx.invoke(self.callback, **ctx.params)                                │
│   1405                                                                                           │
│                                                                                                  │
│ /usr/local/python/3.10.4/lib/python3.10/site-packages/click/core.py:760 in invoke                │
│                                                                                                  │
│    759 │   │   │   with ctx:                                                                     │
│ ❱  760 │   │   │   │   return __callback(*args, **kwargs)                                        │
│    761                                                                                           │
│                                                                                                  │
│ /usr/local/python/3.10.4/lib/python3.10/site-packages/click/decorators.py:26 in new_func         │
│                                                                                                  │
│    25 │   def new_func(*args, **kwargs):  # type: ignore                                         │
│ ❱  26 │   │   return f(get_current_context(), *args, **kwargs)                                   │
│    27                                                                                            │
│                                                                                                  │
│ /usr/local/python/3.10.4/lib/python3.10/site-packages/modal/cli/run.py:115 in f                  │
│                                                                                                  │
│   114 │   │   │   else:                                                                          │
│ ❱ 115 │   │   │   │   func(*args, **kwargs)                                                      │
│   116 │   │   │   if app.function_invocations == 0:                                              │
│                                                                                                  │
│ /workspaces/developer/debugger.py:40 in main                                                     │
│                                                                                                  │
│   39   prompt += "\n\nGive me ideas for what could be wrong and what fixes to do in which fil    │
│ ❱ 40   res = generate_response.call(system, prompt, model)                                       │
│   41   # print res in teal                                                                       │
│                                                                                                  │
│ /usr/local/python/3.10.4/lib/python3.10/site-packages/synchronicity/synchronizer.py:443 in       │
│ proxy_method                                                                                     │
│                                                                                                  │
│                 ...Remote call to Modal Function (ta-WXXcuOtyiK1GQo8XrNPZ8k)...                  │
│                                                                                                  │
│ /root/debugger.py:80 in generate_response                                                        │
│                                                                                                  │
│ ❱ 80 response = openai.ChatCompletion.create(**params)                                           │
│                                                                                                  │
│                                                                                                  │
│ /usr/local/lib/python3.10/site-packages/openai/api_resources/chat_completion.py:25 in create     │
│                                                                                                  │
│ ❱ 25 return super().create(*args, **kwargs)                                                      │
│                                                                                                  │
│                                                                                                  │
│ /usr/local/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py:153 │
│ in create                                                                                        │
│                                                                                                  │
│ ❱ 153 response, _, api_key = requestor.request(                                                  │
│                                                                                                  │
│                                                                                                  │
│ /usr/local/lib/python3.10/site-packages/openai/api_requestor.py:230 in request                   │
│                                                                                                  │
│ ❱ 230 resp, got_stream = self._interpret_response(result, stream)                                │
│                                                                                                  │
│                                                                                                  │
│ /usr/local/lib/python3.10/site-packages/openai/api_requestor.py:624 in _interpret_response       │
│                                                                                                  │
│ ❱ 624 self._interpret_response_line(                                                             │
│                                                                                                  │
│                                                                                                  │
│ /usr/local/lib/python3.10/site-packages/openai/api_requestor.py:687 in _interpret_response_line  │
│                                                                                                  │
│ ❱ 687 raise self.handle_error_response(                                                          │
│                                                                                                  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 7575 tokens. Please reduce the length of the messages

Have GPT-4 API access, running in a DevContainer in Visual Studio Code on Windows 10.

I am not sure how to see exactly what it is outputting to hit this 7575 tokens -- are there any debug steps I can take?

My debug prompt is quite short, it is a short traceback.

[swyxio]

yeah honeslty smol debugger is quite underdeveloped now, i think i know how to solve this but i was leaving it to another day. i might actually split it out to a separate project since it is so useful to run on existing codebases. (aka i'd love to package it up into a binary you can download or brew install)

[swyxio]

basically i plan to chunk the code and put it in chroma, and then bundle it up in to a standalone binary #13

[localjo]

I created an app with:

modal run main.py --prompt "A minimal React app that just displays the text 'Hello, World' in the browser"

It had a small error preventing it from running, so I put that into the debugger:

modal run debugger.py --prompt "Module not found: Error: Can't resolve './App' in '/Users/localjo/smol-dev/generated/src'"

And I got "InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 2188582 tokens. Please reduce the length of the messages." 🤯

Is it trying to put all of the node_modules dependencies in?

[BlackNipples]

is this the following error?

InvalidRequestError: This model's maximum context length is 4097 tokens. However, you requested 4151 tokens (151 in the messages, 4000 in the completion). Please reduce the length of the messages or completion.

[swyxio]

yeah sorry guys this is a bad pain point.

would take a PR to:

1. check token length (use tiktoken) and report a nice friendly error saying we are working on it
2. for @localjo yeah we need to respect gitignore and not check the node_modules

[BlackNipples]

should the program divide the prompt in chunks? just like auto-gpt?

[swyxio]

@BlackNipples i'm tackling the problem a different way - more details in the body of this post https://www.latent.space/p/function-agents#details