Do not pass stream_options param to AzureOpenai

jackmpcollins / magentic

Seamlessly integrate LLMs as Python functions

https://magentic.dev/

MIT License

2.04k stars 100 forks source link

Do not pass stream_options param to AzureOpenai #261

Closed jackmpcollins closed 4 months ago

jackmpcollins commented 4 months ago

According to usage in https://github.com/borh/hachidaishu-translation , AzureOpenAI does yet support stream_options which is used to request usage info from OpenAI. For the moment this should be left unset when using the AzureOpenAI client.

https://github.com/borh/hachidaishu-translation/blob/d1fc5e51365f3a10a4e90d906f00a7070e4e1002/src/hachidaishu_translation/gen.py#L202-L203

Related issues

cc @borh

borh commented 4 months ago

Thanks! I tried your PR against my code without the workaround and it worked.

jackmpcollins commented 4 months ago

@borh Thanks for testing! I haven't been using Azure myself so if you find any more bugs or improvements please file an issue or PR.

I've made an issue to add tests against Azure. It might even be worth making a separate AzureChatModel if there's enough difference with the OpenaiChatModel. https://github.com/jackmpcollins/magentic/issues/266

jackmpcollins commented 4 months ago

@borh The fix is released now in https://github.com/jackmpcollins/magentic/releases/tag/v0.28.0 Please let me know how that works for you