sashabaranov / go-openai

OpenAI ChatGPT, GPT-3, GPT-4, DALL·E, Whisper API wrapper for Go
Apache License 2.0
9.25k stars 1.42k forks source link

Feature Request: Support extra_body Parameter for Advanced Model Features #898

Open wqj97 opened 5 days ago

wqj97 commented 5 days ago

Background vLLM currently supports various model features through configuration parameters, but lacks support for passing additional model-specific parameters through extra_body, which is particularly important for features like structured output. https://github.com/vllm-project/vllm/blob/v0.6.0/vllm/engine/arg_utils.py#L276

Current OpenAI implementation

completion = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Generate a user profile"}],
    extra_body={
        "guided_json": Test.schema_json,
        "guided_decoding_backend": "lm-format-enforcer"
    }
)

Proposed implementation

resp, err := integrations.LLMClient.Client.CreateChatCompletion(
        ctx,
        openai.ChatCompletionRequest{
            Model: "...",
            Messages: []openai.ChatCompletionMessage{
                ...
            },
            ExtraBody: map[string]any{
                ....
            },
        },
    )