hongkai-neu commented 6 days ago

Add streaming support for o1 models

Issue #4759 : Enhancement: Add streaming support for o1-preview and o1-mini

Summary

This PR adds streaming support for o1 models by removing the if condition that was checking if the model was o1 before deleting the modelOptions.stream and modelOptions.stop parameters in the OpenAIClient.js file.

Change Type

[x] New feature (non-breaking change which adds functionality)

Testing

Backend unit tests are passed: npm run test:api

clip

Checklist

[x] My code adheres to this project's style guidelines
[x] I have performed a self-review of my own code

hongkai-neu commented 5 days ago

Streaming with o1-mini and o1-preview work fine with OpenAI/OpenRouter but haven’t been tested with Azure OpenAI due to no API access.