We've been seeing some instability in production with the runtime creation of API keys approach. This is because of race conditions where one thread creates an API key and then at a later time another thread needs to re-create the key (e.g. due to the worker or pod crashing and being restarted) thus invalidating the first one.
Also:
Increase minimum number of pods in Azure deployment to increase availability.
We've been seeing some instability in production with the runtime creation of API keys approach. This is because of race conditions where one thread creates an API key and then at a later time another thread needs to re-create the key (e.g. due to the worker or pod crashing and being restarted) thus invalidating the first one.
Also:
server/utils.pyfile.