search

betalgo / openai

OpenAI .NET sdk - Azure OpenAI, ChatGPT, Whisper, and DALL-E
https://betalgo.github.io/openai/
MIT License
2.85k stars 513 forks source link

Support for RunPod vLLM serverless endpoint #467

Closed carlfm01 closed 5 months ago

carlfm01 commented 6 months ago

Is your feature request related to a problem? Please describe. It can be really useful to replace the enpoint with a RunPod serverless based on vLLM.

Describe the solution you'd like Use url of RunPod vLLM for completion.

Additional context RunPod uses "jobs" it may require management of those jobs' statuses, docs here:

https://docs.runpod.io/docs/serverless-endpoint-urls

kayhantolga commented 5 months ago

Hi, I don't think adding support RunPod to main library makesense, I would like to keep focus on azureAI and openAI. if you would like to add support in utilities package https://www.nuget.org/packages/Betalgo.OpenAI.Utilities/ feel free to create an PR, unforutnalty I don't have time to support it unless high demand for it. Hi, I don't think it makes sense to add support for RunPod to the main library. I would prefer to keep the focus on AzureAI and OpenAI. However, if you would like to add support in the utilities. package, you can create a pull request. Unfortunately, I don't have time to support it unless there is high demand for it.