search

entropy-research / Devon

Devon: An open-source pair programmer
GNU Affero General Public License v3.0
3.27k stars 263 forks source link

Prompt for File Tree Display Permission #92

Open 99Gens opened 4 months ago

99Gens commented 4 months ago

Devon burns excessive amounts of tokens reading a non abbreviated version of the tree. So there either needs to be a light LLM client side which abbreviates it or a always-active request to the user to perform the action, allowing the user to submit their own abbreviated version. With these rate limits, allowing Devon to run known tasks that can eat up a ton of usage is illogical. I see the pair programmers of the future using lite llms on their local machines to reduce the token exposure going up. For the specific task of abbreviating a file tree, you might not even need a full LLM. A simpler natural language processing (NLP) model or even a well-crafted rule-based system could potentially handle this task effectively.

99Gens commented 4 months ago
Screenshot 2024-07-18 at 1 55 33 PM

It literally opens it when the program starts before even saying hello. Burning 80,000 tokens. It's like its trying to punish us. This is a fresh start of the application running npx devon-ui and completely new chat. This is a serious problem. With the existing rate limits imposed by Claude burning that many tokens for no reason is severely debilitating.

Mihir1003 commented 4 months ago

Hey thanks a lot for pointing it out. I will make sure to add a prompt where the user can provide an abbreviated version. Will release this on Saturday!

You're right it does waste a lot of unnecessary tokens. Something we could do potentially is use something like haiku to summarize is file display tree is pretty big.

Something look into would also be being able to ignore certain folders and files.