Support anyscale endpoint

[kcrandall]

Anyscale endpoints support a similar REST format and structure to openai and they even work natively with node and python openai packages. This package should work with anyscale if you just allow editing of the base endpoint when setting up the client or as a method call to a client.

Also would need to add an option to pass a model string instead of predefined enum.

https://docs.endpoints.anyscale.com

For full support you'd need to add ability to add a few custom parameters that any scale supports, but this isn't as important.

https://docs.endpoints.anyscale.com/guides/migrate-from-openai

[kcrandall]

I see the basedomain is exposed in the options so that's already built in as a feature (unless you want to add anyscale as a provider) I guess only thing needed would be to allow passing strings to the model for things like llama etc

[kcrandall]

This is fully supported actually but it might be nice to add an offical provider for it in the code base.

here is code to of how to use anyscale:

public static async IAsyncEnumerable<string> InvokeStreamAsync(string model = "mistralai/Mistral-7B-Instruct-v0.1")
        {
            static string BaseUrl = "https://api.endpoints.anyscale.com/v1";
            var openAiService = new OpenAIService(new OpenAiOptions()
            {
                ApiKey = Token,
                BaseDomain = BaseUrl
            });

            var completionResult = openAiService.ChatCompletion.CreateCompletionAsStream(new ChatCompletionCreateRequest
            {
                Messages = new List<ChatMessage>
                {
                    ChatMessage.FromSystem("You are a helpful assistant."),
                    ChatMessage.FromUser("Who won the world series in 2020?"),
                    ChatMessage.FromAssistant("The Los Angeles Dodgers won the World Series in 2020."),
                    ChatMessage.FromUser("Where was it played?")
                },
                Model = model,
                MaxTokens = 50//optional
            });
            await foreach (var completion in completionResult)
            {
                if (completion.Successful)
                {
                    //Debug.WriteLine("completion"+JsonSerializer.Serialize(completion));
                    yield return completion.Choices.FirstOrDefault()?.Message.Content;
                }
                else
                {

                    if (completion.Error == null)
                    {
                        throw new Exception("Unknown Error");
                    }
                    Debug.WriteLine($"{completion.Error.Code}: {completion.Error.Message}");
                }
            }

        }