gptscript-ai / gptscript

Build AI assistants that interact with your systems
https://gptscript.ai
Apache License 2.0
3.02k stars 267 forks source link

Prompted for entering OpenAI API token by `gpt4-v-vision` and `dalle-image-generation` even when `sys.openai` credential is already present. #482

Closed sangee2004 closed 4 months ago

sangee2004 commented 4 months ago

gptscript version v0.8.1+da33973d

Steps to reproduce the problem:

  1. Have OpenAI API token already set (sys.openai credential created) by executing any gptscript.
  2. Execute the following script that uses gpt4-v-vision and dalle-image-generation tools.
    
    Tools:  github.com/gptscript-ai/gpt4-v-vision, github.com/gptscript-ai/dalle-image-generation, sys.download

Generate an image of cow and download it in cow.png Describe the image in file://./cow.png like a kindergarten teacher would.


3. User is prompted to enter OpenAI API token by both  `gpt4-v-vision` and `dalle-image-generation` tools even when this token is already set as part of `sys.openai` credential creation (from step 1).

gptscript --disable-cache testimagevision.gpt
12:32:09 started [main] 12:32:09 sent [main] 12:32:10 started [image-generation(2)] [input={"prompt": "a cow", "number": "1"}] 12:32:10 started [sys.download(3)] [input={"url": "https://example.com/cow.png", "location": "cow.png"}] 12:32:10 sent [sys.download(3)] 12:32:10 download [https://example.com/cow.png] to [cow.png] 12:32:10 started [credential: github.com/gptscript-ai/dalle-image-generation/credential] 12:32:10 sent [credential: github.com/gptscript-ai/dalle-image-generation/credential] 12:32:10 ended [sys.download(3)] [output=invalid status code [500] downloading [https://example.com/cow.png]: 500 Internal Server Error] Please enter your OpenAI API token. ? token ***** 12:33:09 ended [credential: github.com/gptscript-ai/dalle-image-generation/credential] [output={\"env\": {\"OPENAI_API_KEY\": \"sk-*\"}}] 12:33:09 sent [image-generation(2)] 12:33:23 ended [image-generation(2)] [output=['https://oaidalleapiprodscus.blob.core.windows.net/private/org-UZ21HIS1j93qUiZVWBOGePHu/user-nfhF7k...] 12:33:23 continue [main] 12:33:23 sent [main] 12:33:32 started [sys.download(5)] [input={"url":"https://oaidalleapiprodscus.blob.core.windows.net/private/org-UZ21HIS1j93qUiZVWBOGePHu/user-nfhF7kaasrYgX69n1He52KF9/img-JzZKGjPtrtOAQ5ms55byrgkg.png?st=2024-06-11T18%3A33%3A22Z&se=2024-06-11T20%3A33%3A22Z&sp=r&sv=2023-11-03&sr=b&rscd=inline&rsct=image/png&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skt=2024-06-11T18%3A51%3A10Z&ske=2024-06-12T18%3A51%3A10Z&sks=b&skv=2023-11-03&sig=m8zPdyQEN%2B5eMFumqROLKURLSDU4Cod9QabGpY9cJ0Q%3D","location":"cow.png"}] 12:33:32 sent [sys.download(5)] 12:33:32 download [https://oaidalleapiprodscus.blob.core.windows.net/private/org-UZ21HIS1j93qUiZVWBOGePHu/user-nfhF7kaasrYgX69n1He52KF9/img-JzZKGjPtrtOAQ5ms55byrgkg.png?rscd=inline&rsct=image%2Fpng&se=2024-06-11T20%3A33%3A22Z&sig=m8zPdyQEN%2B5eMFumqROLKURLSDU4Cod9QabGpY9cJ0Q%3D&ske=2024-06-12T18%3A51%3A10Z&skoid=6aaadede-4fb3-4698-a8f6-684d7786b067&sks=b&skt=2024-06-11T18%3A51%3A10Z&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skv=2023-11-03&sp=r&sr=b&st=2024-06-11T18%3A33%3A22Z&sv=2023-11-03] to [cow.png] 12:33:32 ended [sys.download(5)] [output=Downloaded https://oaidalleapiprodscus.blob.core.windows.net/private/org-UZ21HIS1j93qUiZVWBOGePHu/us...] 12:33:32 continue [main] 12:33:32 sent [main] 12:33:35 started [vision(6)] [input={"images":"file://./cow.png","prompt":"Describe the image like a kindergarten teacher would."}] 12:33:35 started [credential: github.com/gptscript-ai/gpt4-v-vision/credential] 12:33:35 sent [credential: github.com/gptscript-ai/gpt4-v-vision/credential] Please enter your OpenAI API token. ? token *** 12:34:06 ended [credential: github.com/gptscript-ai/gpt4-v-vision/credential] [output={\"env\": {\"OPENAI_API_KEY\": \"sk-*\"}}] 12:34:06 sent [vision(6)] 12:34:19 ended [vision(6)] [output={\n \"id\": \"chatcmpl-9Z1Xv5M9by6w4lA34VOUnPj5zbepX\",\n \"object\": \"chat.completion\",\n \"created\"...] 12:34:19 continue [main] 12:34:20 sent [main] 12:34:24 ended [main] [output=Alright, kiddos, let's look at this fun picture together! We see a lovely cow standing in a green fi...] 12:34:24 usage [total=5417] [prompt=4834] [completion=583]

OUTPUT:

Alright, kiddos, let's look at this fun picture together! We see a lovely cow standing in a green field. The cow has a white coat with black spots, which make it look like it's wearing a patterned dress! It has big, gentle eyes and a cute, pink nose that looks soft to touch. The cow's ears are sticking out and it seems very curious, maybe wondering about us just as much as we're wondering about it.

In the background, there's a wooden fence, making sure our cow friend stays safe in the field. The sky is so blue, just like when we draw with our crayons, and there are a few soft, fluffy clouds floating by. It seems like a beautiful sunny day, perfect for our cow to enjoy some fresh, green grass. What sounds do you think the cow makes? That's right, "Moo!" Now, let's imagine we're playing in the field with our cow friend, feeling the cool breeze, and watching the clouds go by. Isn't it fun?



**Expected Behavior:**
There should be no need for entering the  OpenAI API token when it is already present.
g-linville commented 4 months ago

I updated these to just use the generic credential tool and request the sys.openai credential, so this is fixed.

sangee2004 commented 4 months ago

Tested with latest gptscript version - v0.0.0-dev-ac679cd0-dirty Not prompted for entering OpenAI API token by gpt4-v-vision and dalle-image-generation tools. when sys.openai is already present.