Lab 05 - Inconsistency result for Image Generation

[VincentK16]

Module: 05

Lab/Demo: 05

Task: 00

Step: 00

Description of issue: While running the Lab 5 code after configuring the .env with my Azure OpenAI resources and key deployed in East US, I often get the return as

'data'

See attached:

it works sometimes but not all the time.

Repro steps:

1. Download the Lab files
2. Configure the key and endpoints for East US Azure OpenAI resources
3. Run the code

[VincentK16]

Thanks for the prompt response. I was not able to download the file.

[ivorb]

@VincentK16 interesting, and you had no code changes between those runs? I'll see if I can repro the issue - it might be a latency in a newly deployed model (which I can add a note for). If you or anyone reading this issue hits the same thing, please report here - would like to know how often this happens for folks.

[VincentK16]

@VincentK16 interesting, and you had no code changes between those runs? I'll see if I can repro the issue - it might be a latency in a newly deployed model (which I can add a note for). If you or anyone reading this issue hits the same thing, please report here - would like to know how often this happens for folks.

Hey @ivorb, my another colleague got this happening when during the delivery. Do you happen to reproduce this? Wondering what's wrong for this.

Thanks!

[ivorb]

@VincentK16 was the model deployed very shortly before running the code for both you and your colleague? Or was this at least a few minutes after deploying a model? getting "data" back as the response with no errors posted to the console makes it seem like either the model wasn't ready to respond, or the API call returned without a completed response. Did you have a solid internet connection?

[VincentK16]

@VincentK16 was the model deployed very shortly before running the code for both you and your colleague? Or was this at least a few minutes after deploying a model? getting "data" back as the response with no errors posted to the console makes it seem like either the model wasn't ready to respond, or the API call returned without a completed response. Did you have a solid internet connection?

I was using my old deployed model, and it looks to me happening as well. I do have a solid internet connection. What do you think will be the possible case for this? Thanks!

[ivorb]

I tested this extensively, and think it's an issue of hitting the API too often. If you keep sending requests, the API rejects it in accordance to rate limits.

I'm guessing you saw it return with "data" faster than a normal API call would take? If you add a print statement of the raw response right under the requests.post call on line 26, the times it returns with 'data' instead of a real response will have a code of 429.

Rate limits for dalle3 is something like 2 capacity units (6 calls per min) however they aren't strictly timed, so there's a little leeway (you might be able to submit more than 6 in a given minute, and might get limited to less than 6 the next)

The solution is to just wait a little bit - I've also see it perform better when you change the input (don't just give the prompt "hello" over and over) as I've also seen the API reject subsequent calls with the same prompt from the same source to prevent unnecessary overload.