search

gptscript-ai / clio

Your friendly and safe CLI Copilot
Apache License 2.0
202 stars 22 forks source link

Clio exits/crashes when rate limit is hit #13

Open davelowelarsson opened 1 month ago

davelowelarsson commented 1 month ago

I've been testing out Clio and I think I found something that might be improved

When having a long conversation I sometimes end up with this rate limit and Clio exits immediately. I was using my own open AI token.

2024/07/24 00:43:59 run encounterd an error: error, status code: 429, message: Rate limit reached for gpt-4o in organization org-0dxf5pZvV7mz1mqmHw8fJ11b on tokens per min (TPM): Limit 30000, Used 18700, Requested 16922. Please try again in 11.244s. Visit https://platform.openai.com/account/rate-limits to learn more. with error output:

I'va added three agents (Argocd, Argo-rollouts and Kustomize) to help me troubleshoot and debug in my clusters. I was trying to debug an issue that that works in one cluster but not the other. So I had a really long conversation where clio first checked and gathered facts for the working cluster and then we started to compare with the malfunctioning one.

All of a sudden I got the rate limit issue and lost my whole context. If there is an error in clio during a chat session it would be helpful to jump back into that same session or somehow continue where the user left off.

(or maybe avoid the rate limit somehow 😅 )

I actually got a similar issue using the Clio "github login" as well, where it exits immediately

2024/07/24 01:11:58 run encounterd an error: error, status code: 400, message: This model's maximum context length is 128000 tokens. However, your messages resulted in 256710 tokens (254735 in the messages, 1975 in the functions). Please reduce the length of the messages or functions. with error output:

ibuildthecloud commented 1 month ago

Oh shoot, that's the central account that's hitting the rate limit. Hmm.... But definitely we should fix not losing the whole converstaion. Sorry about that.

davelowelarsson commented 1 month ago

No worries, I know i'm testing out something not finished :D

I've also started testing Clio with the llama3.1 model using ollama ... but that seems to not work at all as intended. I know you have been focusing on openAI's models, but should I maybe create a new issue around that? it feels like a good use case for Clio.

To be able to un in an airgapped kubernetes environment and still get to troubleshoot and get a hand with debugging cluster. Or if the company just has rules against sending any kind of data to openAI (or outside or europe) etc. etc.

example output

clio --openai-base-url http://localhost:11434/v1 --openai-model llama3.1 --openai-api-key ollama
> Starting clio (version v0.1.2)...

  This is a list of functions that can be used with a web browser, likely a Chrome extension. Here's a breakdown of each function:

  1.  fill :
    • Description: Fills out an input field on a webpage.
    • Parameters:
      •  content : The text to fill into the element.
      •  keywords : A comma-separated list of keywords related to the element (required).
      •  matchTextOnly : An optional parameter that matches elements based on their text content and ignores their attributes
      (default: false).
      •  tabID : The ID of the tab (optional, default: a new tab will be created).
      •  userInput : A description of the element to fill (required).
      ...
      ...