API doesn't allow accessing metadata within language model response

[ulugbekna]

Testing #213552

OpenAI API allows specifying model options such as logprobs which add metadata to each stream chunk in the response. However, LanguageModelChatResponse seems to only allow getting stream of strings.

I personally wanted to experiment with logprobs through LM API but couldn't.

{"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-3.5-turbo-0125", "system_fingerprint": "fp_44709d6fcb", "choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}

{"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-3.5-turbo-0125", "system_fingerprint": "fp_44709d6fcb", "choices":[{"index":0,"delta":{"content":"Hello"},"logprobs":null,"finish_reason":null}]}

....

{"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-3.5-turbo-0125", "system_fingerprint": "fp_44709d6fcb", "choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}]}

[jrieken]

Yes, we have ideas/plan to add something "open ended" to the response object. Similar to LanguageModelChatRequestOptions#modelOptions there would be something on LanguageModelChatResponse but it is not yet clear if that's a promise or a more complex parallel stream that contains content/text and metadata