[BUG]: TokenLimitError

[bigvo]

After 25 iteration on first task, app crashes with error below, if I rerun app with app_id, it just crashes after load, If I rerun with skip to previous to crash step, it executes one command and crashes again.

`---------- GPT PILOT EXITING WITH ERROR ---------- Traceback (most recent call last): File "/usr/src/app/pilot/utils/llm_connection.py", line 169, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/app/pilot/utils/llm_connection.py", line 345, in stream_gpt_completion raise Exception(f"API responded with status code: {response.status_code}. Response text: {response.text}") Exception: API responded with status code: 400. Response text: { "error": { "message": "This model's maximum context length is 8192 tokens. However, your messages resulted in 8337 tokens. Please reduce the length of the messages.", "type": "invalid_request_error", "param": "messages", "code": "context_length_exceeded" } }

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/src/app/pilot/main.py", line 67, in project.start() File "/usr/src/app/pilot/helpers/Project.py", line 134, in start self.developer.start_coding() File "/usr/src/app/pilot/helpers/agents/Developer.py", line 50, in start_coding self.implement_task(i, dev_task) File "/usr/src/app/pilot/helpers/agents/Developer.py", line 114, in implement_task response = convo_dev_task.send_message('development/task/update_task.prompt', result, IMPLEMENT_TASK) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/app/pilot/helpers/AgentConvo.py", line 86, in send_message raise e File "/usr/src/app/pilot/helpers/AgentConvo.py", line 82, in send_message response = create_gpt_chat_completion(self.messages, self.high_level_step, self.agent.project, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/app/pilot/utils/llm_connection.py", line 110, in create_gpt_chat_completion raise e File "/usr/src/app/pilot/utils/llm_connection.py", line 103, in create_gpt_chat_completion response = stream_gpt_completion(gpt_data, req_type, project) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/app/pilot/utils/llm_connection.py", line 216, in wrapper raise TokenLimitError(get_tokens_in_messages_from_openai_error(err_str), MAX_GPT_MODEL_TOKENS) helpers.exceptions.TokenLimitError.TokenLimitError: Token limit error happened with 8337/8192 tokens in messages! --------------------------------------------------`

[bigvo]

App is being run in a Docker container and setting correctly set to: MAX_TOKENS=8192

[Scratch-project]

Did you try to set the limit to 8337 or 8338 ? It seems like the output want to use 8337 and your limit 8192

[bigvo]

Did you try to set the limit to 8337 or 8338 ? It seems like the output want to use 8337 and your limit 8192

I did not, because this limit is set by OpenAI to gpt-4 model

[Scratch-project]

Did you try to set the limit to 8337 or 8338 ? It seems like the output want to use 8337 and your limit 8192

I did not, because this limit is set by OpenAI to gpt-4 model

Are you sure ?

I think the limit is 4k or 10k depending on wether you paid or not.

[bigvo]

The limits you are referring to are related to rate usage limit, but the limit I hit and limit which sets in settings is max token per request of the passed context to gpt-4 on request, it is set correctly in settings, 32K model is unfortunatunately unavailable for me to increase the context.

[Scratch-project]

The limits you are referring to are related to rate usage limit, but the limit I hit and limit which sets in settings is max token per request of the passed context to gpt-4 on request, it is set correctly in settings, 32K model is unfortunatunately unavailable for me to increase the context.

Can you explain it more ? I didn't get it..... What is the difference between your pic and my pic and who have access to gpt-4-32k ?

[bigvo]

The limit you are referring to is about rate of requests per minute in tokens and amount of requests, my reference to context size which may be passed in one request (and error I got because of the context size not rate limit).

FYI as a solution I switched to gpt-4-32k from Openrouter, GPT Pilot support Openrouter out-of-box and it works like a charm.

[Scratch-project]

The limit you are referring to is about rate of requests per minute in tokens and amount of requests, my reference to context size which may be passed in one request (and error I got because of the context size not rate limit).

FYI as a solution I switched to gpt-4-32k from Openrouter, GPT Pilot support Openrouter out-of-box and it works like a charm.

From openrouter ? Is this free ? I'm using gpt-4 as I paid 5 dollars, now I'm trying to use gpt-4-32k and it says I don't have access to it.

[bigvo]

It is not, please refer to their website.

[Scratch-project]

It is not, please refer to their website.

Wanted to ask what Openrouter is ? I mean why do they have Openai models ? What do they do actually ?

[hofmanfr]

Same Error here! Using the VS Code extention and OpenAI

You reached 54.55% of your project generation!

For now, you have created:

48 files

21064 lines of code

Before continuing, GPT Pilot will create some documentation for the project...

Creating README.md

---------- GPT PILOT EXITING WITH ERROR ----------

Traceback (most recent call last):

File "/Users/xxx/work/gpt-pilot/pilot/utils/llm_connection.py", line 172, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^

File "/Users/xxx/work/gpt-pilot/pilot/utils/llm_connection.py", line 353, in stream_gpt_completion raise Exception(f"API responded with status code: {response.status_code}. Response text: {response.text}")

Exception: API responded with status code: 400. Response text: { "error": { "message": "This model's maximum context length is 128000 tokens. However, your messages resulted in 167918 tokens. Please reduce the length of the messages.", "type": "invalid_request_error", "param": "messages", "code": "context_length_exceeded" } }

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/Users/xxx/work/gpt-pilot/pilot/main.py", line 78, in project.start()

File "/Users/xxx/work/gpt-pilot/pilot/helpers/Project.py", line 141, in start self.developer.start_coding()

File "/Users/xxx/work/gpt-pilot/pilot/helpers/agents/Developer.py", line 67, in start_coding self.project.technical_writer.document_project(current_progress_percent)

File "/Users/xxx/work/gpt-pilot/pilot/helpers/agents/TechnicalWriter.py", line 22, in document_project self.create_readme()

File "/Users/xxx/work/gpt-pilot/pilot/helpers/agents/TechnicalWriter.py", line 33, in create_readme llm_response = convo.send_message('documentation/create_readme.prompt', { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/Users/xxx/work/gpt-pilot/pilot/helpers/AgentConvo.py", line 89, in send_message raise e

File "/Users/xxx/work/gpt-pilot/pilot/helpers/AgentConvo.py", line 85, in send_message response = create_gpt_chat_completion(self.messages, self.high_level_step, self.agent.project, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/Users/xxx/work/gpt-pilot/pilot/utils/llm_connection.py", line 113, in create_gpt_chat_completion raise e

File "/Users/xxx/work/gpt-pilot/pilot/utils/llm_connection.py", line 106, in create_gpt_chat_completion response = stream_gpt_completion(gpt_data, req_type, project) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/Users/xxx/work/gpt-pilot/pilot/utils/llm_connection.py", line 219, in wrapper raise TokenLimitError(get_tokens_in_messages_from_openai_error(err_str), MAX_GPT_MODEL_TOKENS)

helpers.exceptions.TokenLimitError.TokenLimitError: Token limit error happened with 167918/8192 tokens in messages!

[Lupus]

Same here... Asked gpt-pilot to create a golang binary, it created initial main.go, ran go mod tidy, which created large go.sum file with hashes which it seems to try to feed back into the LLM.

helpers.exceptions.TokenLimitError.TokenLimitError: Token limit error happened with Token limit error happened with 33002/8192 tokens in messages!