Unexpected id when stream=True

[mahaddad]

Hey aviary team. The v0.2.0 release is looking great, nice work!

I had a question about the response ID for streaming.

Encountered Behavior

When calling /chat/completions with stream=True each response has a unique id

Expected Behavior

Expect that each response has the same id. This is how OpenAI formats the response.

Is it possible to have the responses have the same id? If not, is there a suggested way to group the response streams?

See below for side-by-side examples of aviary response and OpenAI response chat_completion = openai.ChatCompletion.create( model=model_name, messages=[{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Say 'test me please'."}], temperature=0.7, stream=stream )

{ "id": "meta-llama/Llama-2-7b-chat-hf-ceb3f770-4897-4aa0-bb7b-b33bf4cbb821", "object": "text_completion", "created": 1692221192, "model": "meta-llama/Llama-2-7b-chat-hf", "choices": [ { "delta": { "role": "assistant" }, "index": 0, "finish_reason": null } ], "usage": null } { "id": "meta-llama/Llama-2-7b-chat-hf-7ccf1294-0371-4fbb-a143-306029eafa39", "object": "text_completion", "created": 1692221192, "model": "meta-llama/Llama-2-7b-chat-hf", "choices": [ { "delta": { "content": "Of" }, "index": 0, "finish_reason": null } ], "usage": null } { "id": "meta-llama/Llama-2-7b-chat-hf-1ca78f25-1a41-4d5f-92eb-833997192896", "object": "text_completion", "created": 1692221192, "model": "meta-llama/Llama-2-7b-chat-hf", "choices": [ { "delta": { "content": " course" }, "index": 0, "finish_reason": null } ], "usage": null } { "id": "meta-llama/Llama-2-7b-chat-hf-cee49ab7-fb78-4dd6-97bd-2814e5d48159", "object": "text_completion", "created": 1692221192, "model": "meta-llama/Llama-2-7b-chat-hf", "choices": [ { "delta": { "content": "!" }, "index": 0, "finish_reason": null } ], "usage": null } { "id": "meta-llama/Llama-2-7b-chat-hf-25ef4544-1d92-44f1-bc2d-dd8225f24efd", "object": "text_completion", "created": 1692221192, "model": "meta-llama/Llama-2-7b-chat-hf", "choices": [ { "delta": { "content": " *" }, "index": 0, "finish_reason": null } ], "usage": null } { "id": "meta-llama/Llama-2-7b-chat-hf-4bb5200b-90c7-4541-b33a-37c422bf80cb", "object": "text_completion", "created": 1692221192, "model": "meta-llama/Llama-2-7b-chat-hf", "choices": [ { "delta": { "content": "test" }, "index": 0, "finish_reason": null } ], "usage": null } { "id": "meta-llama/Llama-2-7b-chat-hf-1fa3c651-f4a8-4101-81fb-b8113f935931", "object": "text_completion", "created": 1692221192, "model": "meta-llama/Llama-2-7b-chat-hf", "choices": [ { "delta": { "content": " me" }, "index": 0, "finish_reason": null } ], "usage": null } { "id": "meta-llama/Llama-2-7b-chat-hf-3eeadcd7-f8fc-44a1-b3cc-36f751dfc7e5", "object": "text_completion", "created": 1692221192, "model": "meta-llama/Llama-2-7b-chat-hf", "choices": [ { "delta": { "content": " please" }, "index": 0, "finish_reason": null } ], "usage": null } { "id": "meta-llama/Llama-2-7b-chat-hf-6be0060b-a46e-4553-963a-dd7f43b1d809", "object": "text_completion", "created": 1692221192, "model": "meta-llama/Llama-2-7b-chat-hf", "choices": [ { "delta": { "content": "*" }, "index": 0, "finish_reason": null } ], "usage": null } { "id": "meta-llama/Llama-2-7b-chat-hf-c73f0096-1e6a-4f7f-803f-8603d70489d9", "object": "text_completion", "created": 1692221192, "model": "meta-llama/Llama-2-7b-chat-hf", "choices": [ { "delta": {}, "index": 0, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 35, "completion_tokens": 9, "total_tokens": 44 } }

{ "id": "chatcmpl-7oIKTj4FV7OH8tmG9DUnxpjlSTH2S", "object": "chat.completion.chunk", "created": 1692221213, "model": "gpt-3.5-turbo-0613", "choices": [ { "index": 0, "delta": { "role": "assistant", "content": "" }, "finish_reason": null } ] } { "id": "chatcmpl-7oIKTj4FV7OH8tmG9DUnxpjlSTH2S", "object": "chat.completion.chunk", "created": 1692221213, "model": "gpt-3.5-turbo-0613", "choices": [ { "index": 0, "delta": { "content": "Test" }, "finish_reason": null } ] } { "id": "chatcmpl-7oIKTj4FV7OH8tmG9DUnxpjlSTH2S", "object": "chat.completion.chunk", "created": 1692221213, "model": "gpt-3.5-turbo-0613", "choices": [ { "index": 0, "delta": { "content": " me" }, "finish_reason": null } ] } { "id": "chatcmpl-7oIKTj4FV7OH8tmG9DUnxpjlSTH2S", "object": "chat.completion.chunk", "created": 1692221213, "model": "gpt-3.5-turbo-0613", "choices": [ { "index": 0, "delta": { "content": " please" }, "finish_reason": null } ] } { "id": "chatcmpl-7oIKTj4FV7OH8tmG9DUnxpjlSTH2S", "object": "chat.completion.chunk", "created": 1692221213, "model": "gpt-3.5-turbo-0613", "choices": [ { "index": 0, "delta": { "content": "." }, "finish_reason": null } ] } { "id": "chatcmpl-7oIKTj4FV7OH8tmG9DUnxpjlSTH2S", "object": "chat.completion.chunk", "created": 1692221213, "model": "gpt-3.5-turbo-0613", "choices": [ { "index": 0, "delta": {}, "finish_reason": "stop" } ] }

[Yard1]

Sounds like a reasonable request! cc @avnishn