search

timoklimmer / powerproxy-aoai

Monitors and processes traffic to and from Azure OpenAI endpoints.
MIT License
83 stars 23 forks source link

Feature Request: Explicitly log model, model version and API version #92

Closed krohm closed 1 week ago

krohm commented 2 weeks ago

Hi, not an issue, but a feature request: Is it possible to log additionally to the Calls the Model, Model Version and API-Version that was used by the call?

timoklimmer commented 2 weeks ago

Hi @krohm -- model and model versions can be logged indirectly already today by configuring virtual deployments. Once virtual deployments are used, the logs contain a standin deployment, which is the name of the actual deployment used. From that, you can derive model and model version. I will check if there is a better way to log model and model version.

Tracking the API version is also a good idea. I will look into it and let you know once implemented.

timoklimmer commented 1 week ago

Hi @krohm, I have added the API version to the logs and released a new version. Logging the model and model version is a bit more involved, hence won't implement for now. If you need the model and model version, I suggest to name your deployments accordingly. Since requests will use the virtual deployment name, applications won't need to be changed with new model/versions. Alternatively, you can lookup the model/version in the details of the respective deployments.