grafana / grafana-llm-app

Plugin to easily allow LLM based extensions to grafana
Apache License 2.0
54 stars 4 forks source link

Does not work with free OpenAI Account - should use gpt-3.5-turbo instead #212

Open schenklklopfer opened 7 months ago

schenklklopfer commented 7 months ago

During configuration the LLM plugin prompts after click on "Safe & Test":

OpenAI health check succeeded!
Models
gpt-3.5-turbo: OK
gpt-4: Error: unexpected status code: 404: { "error": { "message": "The model `gpt-4` does not exist or you do not have access to it. Learn more: https://help.openai.com/en/articles/7102672-how-can-i-access-gpt-4.", "type": "invalid_request_error", "param": null, "code": "model_not_found" } } 

The link says: If you're a Pay-As-You-Go customer and you've made a successful payment of $1 or more, you'll be able to access the [GPT-4 API](https://platform.openai.com/docs/models/gpt-4) (8k).

So the old gpt-3.5-turbo should work fine.

But the plugin still tries to use gpt-4.

Once I try to generate a description the page tells me:

Failed to generate content using OpenAI 
Please try again or if the problem persists, contact your organization admin.

And the browser debugger says:

Error: proxy: stream: eventsource.SubscribeWithRequest: https://api.openai.com/v1/chat/completions: error 404: {
    "error": {
        "message": "The model `gpt-4` does not exist or you do not have access to it. Learn more: https://help.openai.com/en/articles/7102672-how-can-i-access-gpt-4.",
        "type": "invalid_request_error",
        "param": null,
        "code": "model_not_found"
    }
}

In this case gpt-3.5-turbo should be used instead.

jtheory commented 4 months ago

Thanks for reporting this!

Confirmed issue; we have a design doc on a better approach (both for supporting more providers, and resolving this limitation) and have scheduled this project for the upcoming quarter (starting May 1st).