Context is consuming 1,000,000 plus tokens a day.

jasonabuck commented 2 months ago

Before submitting your bug report

[ ] I believe this is a bug. I'll try to join the Continue Discord for questions
[ ] I'm not able to find an open issue that reports the same bug
[ ] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: MAC
- Continue: 
- IDE: VS Code
- Model: Claude 3.5 Sonnet
- config.json:

Description

I am trying to figure out what I am doing wrong. Are you supposed to start a new session with each request? One hour into using Continue on day three, two page updates and all of my daily API limit with Anthropic is exceeded.

You should provide a warning that the request will use N tokens. Do you want to continue? Submitting via API is different from submitting via ChatGPT or even the PROJECTS portion on Claude. I will now have to stop using Continue, as the value of having it in my IDE vs. cutting and pasting from my paid subscriptions is not beneficial.

To reproduce

No response

Log output

No response

TyDunn commented 2 months ago

@jasonabuck I'm sorry to hear that this has been such a frustrating experience. It is surprising that you hit your daily API limit so quickly. What is your limit set to currently? Also, can you describe how you've used Continue in a bit more depth (e.g. are you always using @codebase, highlighting entire files as context every time, never starting a new session ever, etc)?

Generally, you'll want to press cmd/ctrl+L to start a new session and reset the context when you switch tasks. If you don't, then you'll be sending a lot of irrelevant context to Claude Sonnet 3.5 every call, which is not only expensive but likely also often makes the reply you receive worse

TyDunn commented 2 months ago

You can also see what context is sent to the model on each request by

Opening the terminal in VS Code
Selecting OUTPUT from the top level tabs
Selecting Continue - LLM Prompt/Completion from the drop down

continuedev / continue