Context length exceeded. High token consumption

[ChenZixinn]

Environment:

• Model version: GPT-4o-2024-08-06
• Issue context: When importing multiple files, an error occurs indicating that the token limit has been exceeded.

Observed behavior:

1. Multiple interactions needed for file import:

• When importing a single file, it often requires several interactions (approximately 6-7 attempts) to succeed, which significantly increases token consumption.
  2. Large data volume during file import:
• During file import, the system sends the first three rows of data. However, when the data file contains many columns, the size of the data sent is quite large, leading to high token usage and potentially exceeding the token limit.

Expected behavior:

• Optimized token consumption by reducing the number of interactions and the size of the data being sent.

[gregnr]

Thanks for the issue @ChenZixinn. Do you have any sample files you could provide to help reproduce your two observations?

[ChenZixinn]

Thank you for your response.

I believe I’ve temporarily resolved the issues. The first problem was related to the CSV file, where some data fields were empty, causing failures and constant retries. By formatting the CSV file—such as filling empty rows with zeros or other values—it reduced the errors.

The second problem was likely caused by the first one, and as I continuously imported the file, it resulted in a large message context. However, setting a maxMessageContext seems to resolve the issue.

I’m sorry I can’t provide the data for reference, but here are two areas for optimization:

1.  Avoiding errors during CSV imports: If there are empty rows or inconsistent data types, it’s easy for errors to occur.
2.  Reducing the logging of non-essential information or simplifying the logs to minimize token consumption.

[ChenZixinn]

I’m really excited about the Private Alpha deployments, especially the idea of using a serverless PGlite instance outside the browser. I’ve already joined the waitlist and can’t wait to get access! Looking forward to hearing more about it soon.