search

matrixgpt / matrix-chatgpt-bot

Talk to ChatGPT via any Matrix client!
GNU Affero General Public License v3.0
234 stars 63 forks source link

Max Prompt Token Usage #140

Closed ericdrgn closed 1 year ago

ericdrgn commented 1 year ago

I am seeing all requests through the bot have max token usage on prompts. Even making a simple prompt of "hi" triggers a prompt token usage of 3000 tokens. Running the same thing through a simple terminal curl request results in a total of 8 tokens used. What's going on? This wasn't happening yesterday.

ericdrgn commented 1 year ago

Confirmed usage in playground is also working fine, it seems the only time token usage is off the charts for a prompt of "hi" is when using this matrix bot. I have attached a screenshot of the usage displayed in the OpenAI dashboard. I ran a prompt of "hi" through the OpenAI Playground, then the Matrix bot, then through a standard curl command. Is no one else seeing this issue? I haven't changed anything and looking through the history of requests it started this morning.

image

ericdrgn commented 1 year ago

Nevermind I think I found the issue and it was due to context being set to both.