microsoft / prompty

Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty is an asset class and format for LLM prompts designed to enhance observability, understandability, and portability for developers.
https://prompty.ai
MIT License
381 stars 24 forks source link

VSCode extension setttings are not read correctly #49

Open Godfather95 opened 1 month ago

Godfather95 commented 1 month ago

I am using the following configuration in settings.json:

{
      "name": "default",
      "type": "azure_openai",
      "azure_deployment": "gpt-35-turbo",
      "api_version": "2023-12-01-preview",
      "azure_endpoint": "${env:AZURE_OPENAI_ENDPOINT}",
      "api_key": "${env:AZURE_OPENAI_API_KEY}"
},
{
      "name": "gpt-3.5-turbo",
      "type": "openai",
      "api_key": "${env:OPENAI_API_KEY}",
      "organization": "${env:OPENAI_ORG_ID}",
      "base_url": "${env:OPENAI_BASE_URL}"
}

When running a .prompty file, I get the following info message in the Prompty Output:

info] Missing prompty configuration setting: MissingDeployment. Your setting is {"name":"default","type":"azure_openai","api_version":"2023-12-01-preview","azure_endpoint":"https://<EndpointName>.openai.azure.com/","api_key":"<APIKEY>"}

It seems the Extension doesn't even read the "azure_deployment" from the settings.

sethjuarez commented 1 month ago

Uh oh - there is a way to select the settings you want at the bottom: image Have you tried switching these? FWIW, we will be adding defaults in a more prominent place soon.

Godfather95 commented 1 month ago

@sethjuarez Yeah, I tried it, and I even copied the settings to be the "non default". But no matter which model configuration I select, I always get the error.

anuraj commented 1 month ago

For me as well. I tried changing the configuration, but always getting the error. image

abakumovoleg commented 1 month ago

the same problem, it always use type azure_openai even if I set openai instead

perkops commented 1 month ago

I am experiencing the same issue (even with hardcoded values, not being read from environment).

sethjuarez commented 1 month ago

oof - ok - I will look into it - I am adding some features to the extension this week/next. There's a couple of invokers I need to add to make this work

AnaTipps commented 1 month ago

I am getting something related, I changed the user settings to add my deployment and after that I start getting this error image I reverted the changed, uninstall and install again my extension and nothing have worked so far. I am using the API Key Endpoint pair in my .env file

AnaTipps commented 1 month ago

I am getting something related, I changed the user settings to add my deployment and after that I start getting this error image I reverted the changed, uninstall and install again my extension and nothing have worked so far. I am using the API Key Endpoint pair in my .env file

It seems the problem has been resolved. I traced back the changes made and reverted them. The issue appeared to be related to the workspace settings. Once I deleted the .vscode folder that generates when you open the workspace settings file, the problem was resolved.

BartNetJS commented 1 month ago

Same issue here

wayliums commented 3 weeks ago

We read the setting from prompty.currentModelConfiguration.

In some cases VSCode saves to .vscode/settings.json. image

And some cases it's saved to your user levels settings. image

@BartNetJS @AnaTipps @perkops @abakumovoleg @anuraj @Godfather95 , can you let me know what's your case? I'll see how to make this less problematic

MikeYeager commented 3 weeks ago

In my case, aside from multiple configuration files, the endpoint was overridden in the yaml file itself. I copied it from a sample and didn't notice the setting and didn't remove it and it overrode my settings files. A strategy I've used in the past is to have a way to retrieve all of the current settings to see if they're what is expected. If a value isn't what you expect, it's just a matter of tracking down the source of the unexpected value. Also, a document explaining all the possible places values can be read from and in what precedence order is also useful.

MikeStall commented 2 weeks ago

Same issue here