search

royerlab / napari-chatgpt

A napari plugin to process and analyse images with chatGPT!
BSD 3-Clause "New" or "Revised" License
233 stars 25 forks source link

Quick token usage #55

Open billflan1996 opened 1 month ago

billflan1996 commented 1 month ago

Hi, thanks for creating Omega - it is very exciting to use!

Possibly too exciting for myself as I seemed to have burnt through the $7 I loaded up over 2 days. I am guessing I am doing things very inefficiently somehow as I had only used it for a couple of hours each day.

I was using gpt-4-turbo-2024-04-09 as the model, are there any other points I should know about to stretch my tokens further?

Thanks again, Bill

aham41 commented 1 month ago

I had this exact same problem, using the standard gpt4-turbo model I burnt through 12$ in just a single day, making this extension economically unfeasible. Any tips to reduce costs? Other than that it's lovely to be able to iterate analysis through a gpt

royerloic commented 3 weeks ago

Dear @billflan1996 and @aham41

Sorry to hear that! Did you notice any strange behaviour from omega, like going into endless loops, etc... Would you mind posting typical terminal output here to see if there is anything strange. I rarely spend much money with Omega. The cost is typically in the $5.000 / 1M input tokens. Something is definitely wrong here. If anyone else has this problem please add a note to this issue. Terminal output is key for me to understand what is going on. How long (in minutes/hours) are you been using Omega, and how many prompts have you done? For example, when I developed Omega, I spent a total of 13$ in something like 3 weeks... If anything the models models are getting cheaper (not all of them of course). are you seeing omega take a lot of snapshots of napari to 'look' at it? That could be also a reason.

To reduce cost you can of course use cheaper models, and you can also turn off the autofix features that increase the token consumption:

image

What kind of machine are you running Omega on? is this a private well maintained and secure personal laptop, or a 'lab computer' running windows and potentially full of viruses? There is malware out there that will steal OpenAI keys, so beware of that. You should ideally use the 'API key Vault' and set spending limits to be safe. Hopefully that is not the issue here...

Hope this helps,keep us posted if you make sense of it and post the terminal outputs so i can help. I will be doing some testing in the next days to see if I observe anything strange.