Excessive input context, input quota exceeded, .gitignore not respected

[marksolly]

In some cases, the logic which decides which files to provide as context will select a file that is so large it consumes the users entire Anthropic accout input token quota for the day. In my opinion, there is no point providing more than approx 50% of the models max context window, and certainly not more than 100%.

I propose two configuration options to prevent this happening:

1. Claude Dev should respect the .gitignore file unless this is disabled in settings.
2. A user configurable file size limit should be applied. Perhaps including options to truncate or halt with error. 60kb could be a good default limit for Claude Sonnet 3.5 with 100k context window.

In this example, 2.6mb of text was used as input because a hidden directory containing compiler artifacts was selected as context: claude_dev_task_sep-11-2024_12-35-12-pm.md (caution, very large text file)

Resulting error for this and all subsequent tasks:

API Request Failed

429 {"type":"error","error":{"type":"rate_limit_error","message":"Number of request tokens has exceeded your daily rate limit (https://docs.anthropic.com/en/api/rate-limits); see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}

[saoudrizwan]

You can add an 'ignore these files' list in Settings > Custom Instructions so claude does not read particular files into context. Outside of that it's hard for me to think adding a UX for this would help because you have to manually approve files before claude reads them... if you have any good ideas for this pls feel free to add it to the features request board so people can vote on it