Closed harishv-collab closed 5 months ago
Adds an instanceID field in ChatCompletionRequest which can be used to target a specific multi LoRA reserved instance.
Sample curl command: curl -X POST https://api.openai.com/v1/chat/completions -H "Authorization: Bearer $OPENAI_API_KEY" -d '{"messages": [{"role":"user", "content": "how do i tie my shoes?"}], "model":"gpt-3.5-turbo-1106", "instance_id": "gpt-3.5-turbo-1106-instance"}' -H 'Content-Type: application/json'
curl -X POST https://api.openai.com/v1/chat/completions -H "Authorization: Bearer $OPENAI_API_KEY" -d '{"messages": [{"role":"user", "content": "how do i tie my shoes?"}], "model":"gpt-3.5-turbo-1106", "instance_id": "gpt-3.5-turbo-1106-instance"}' -H 'Content-Type: application/json'
There's no option currently to target a specific instance/fleet in OpenAI, especially if using multi LoRA.
Describe your change
Adds an instanceID field in ChatCompletionRequest which can be used to target a specific multi LoRA reserved instance.
Sample curl command:
curl -X POST https://api.openai.com/v1/chat/completions -H "Authorization: Bearer $OPENAI_API_KEY" -d '{"messages": [{"role":"user", "content": "how do i tie my shoes?"}], "model":"gpt-3.5-turbo-1106", "instance_id": "gpt-3.5-turbo-1106-instance"}' -H 'Content-Type: application/json'What problem is this fixing?
There's no option currently to target a specific instance/fleet in OpenAI, especially if using multi LoRA.