openai: add toolcalls support when streaming

ChrisCPoirier commented 3 months ago

PR Checklist

[x] Read the Contributing documentation.
[x] Read the Code of conduct documentation.
[x] Name your Pull Request title clearly, concisely, and prefixed with the name of the primarily affected package you changed according to Good commit messages (such as memory: add interfaces for X, Y or util: add whizzbang helpers).
[x] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. Fixes #123).
[x] Describes the source of new concepts.
[x] References existing implementations as appropriate.
[x] Contains test coverage for new functions.
[x] Passes all golangci-lint checks.

I noticed that after the recent addition of ToolCalls support that streaming with functions/toolCalls is no longer working. I created this PR to add tool calls support to the streaming functionality.

you can see this in action if you call this example: https://github.com/tmc/langchaingo/blob/main/examples/openai-function-call-streaming-example/openai_function_call_example.go

This example with current code will not print this final message. With the change in this PR you will see the correct result

Expected output:

Function call: &{getCurrentWeather {
  "location": "Boston, MA",
  "rationale": "User asked for the current weather in Boston"
}}

devalexandre commented 2 months ago

PR Checklist
* [x]  Read the [Contributing documentation](https://github.com/tmc/langchaingo/blob/main/CONTRIBUTING.md).

* [x]  Read the [Code of conduct documentation](https://github.com/tmc/langchaingo/blob/main/CODE_OF_CONDUCT.md).

* [x]  Name your Pull Request title clearly, concisely, and prefixed with the name of the primarily affected package you changed according to [Good commit messages](https://go.dev/doc/contribute#commit_messages) (such as `memory: add interfaces for X, Y` or `util: add whizzbang helpers`).

* [x]  Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.

* [x]  Provide a description in this PR that addresses **what** the PR is solving, or reference the issue that it solves (e.g. `Fixes #123`).

* [x]  Describes the source of new concepts.

* [x]  References existing implementations as appropriate.

* [x]  Contains test coverage for new functions.

* [x]  Passes all [`golangci-lint`](https://golangci-lint.run/) checks.
I noticed that after the recent addition of ToolCalls support that streaming with functions/toolCalls is no longer working. I created this PR to add tool calls support to the streaming functionality.

you can see this in action if you call this example: https://github.com/tmc/langchaingo/blob/main/examples/openai-function-call-streaming-example/openai_function_call_example.go

This example with current code will not print this final message. With the change in this PR you will see the correct result

Expected output:
Function call: &{getCurrentWeather {
  "location": "Boston, MA",
  "rationale": "User asked for the current weather in Boston"
}}

could you add some example in example folder?

ChrisCPoirier commented 2 months ago

@devalexandre This example already exists and is not working correctly in current state: examples/openai-function-call-streaming-example/openai_function_call_example.go

results without my change (not expected):

go run examples/openai-function-call-streaming-example/openai_function_call_example.go
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:

Results with this change (expected):

go run examples/openai-function-call-streaming-example/openai_function_call_example.go
Received chunk: [{"id":"call_UMEtDyZ0OUspw9fjt0DSNTlO","type":"function","function":{"name":"getCurrentWeather","arguments":""}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"{\n"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" "}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" \""}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"location"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"\":"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" \""}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"Boston"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":","}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" MA"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"\",\n"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" "}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" \""}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"ration"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"ale"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"\":"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" \""}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"User"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" asked"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" for"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" the"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" current"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" weather"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" in"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" Boston"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"\"\n"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"}"}}]
Received chunk:
Function call: &{getCurrentWeather {
  "location": "Boston, MA",
  "rationale": "User asked for the current weather in Boston"
}}

devalexandre commented 2 months ago

@devalexandre This example already exists and is not working correctly in current state: examples/openai-function-call-streaming-example/openai_function_call_example.go

results without my change (not expected):

go run examples/openai-function-call-streaming-example/openai_function_call_example.go
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:
Received chunk:

Results with this change (expected):

go run examples/openai-function-call-streaming-example/openai_function_call_example.go
Received chunk: [{"id":"call_UMEtDyZ0OUspw9fjt0DSNTlO","type":"function","function":{"name":"getCurrentWeather","arguments":""}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"{\n"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" "}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" \""}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"location"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"\":"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" \""}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"Boston"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":","}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" MA"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"\",\n"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" "}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" \""}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"ration"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"ale"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"\":"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" \""}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"User"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" asked"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" for"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" the"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" current"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" weather"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" in"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":" Boston"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"\"\n"}}]
Received chunk: [{"type":"","function":{"name":"","arguments":"}"}}]
Received chunk:
Function call: &{getCurrentWeather {
  "location": "Boston, MA",
  "rationale": "User asked for the current weather in Boston"
}}

very nice :)

douglarek commented 2 months ago

There should be a parameter here to control not outputting function call parameters to the chunk. For behaviors that do not involve hitting a function call, the streaming chunk at this time is very clean and can be restored normally. However, if you want to keep the streaming chunk clean when hitting function calls, it is a bit difficult to obtain it continuously without a better way to filter normal chunk and function call parameters.

For example, I have a function call to calculate exchange rates. When it is hit, the streaming output is:

[{"id":"call_rKVeh7zAY6Sc5c2NeUQAQiIY","type":"function","function":{"name":"getExchangeRate","arguments":""}}]
[{"type":"","function":{"name":"","arguments":"{""}}]
[{"type":"","function":{"name":"","arguments":"currency"}}]
[{"type":"","function":{"name":"","arguments":"_date"}}]
[{"type":"","function":{"name":"","arguments":"":""}}]
[{"type":"","function":{"name":"","arguments":"latest"}}]
[{"type":"","function":{"name":"","arguments":"",""}}]
[{"type":"","function":{"name":"","arguments":"currency"}}]
[{"type":"","function":{"name":"","arguments":"_from"}}]
[{"type":"","function":{"name":"","arguments":"":""}}]
[{"type":"","function":{"name":"","arguments":"JP"}}]
[{"type":"","function":{"name":"","arguments":"Y"}}]
[{"type":"","function":{"name":"","arguments":"",""}}]
[{"type":"","function":{"name":"","arguments":"currency"}}]
[{"type":"","function":{"name":"","arguments":"_to"}}]
[{"type":"","function":{"name":"","arguments":"":""}}]
[{"type":"","function":{"name":"","arguments":"C"}}]
[{"type":"","function":{"name":"","arguments":"NY"}}]
[{"type":"","function":{"name":"","arguments":""}"}}].

When this function is not hit, the streaming output is:

Hello
!
 How
 can
 I
 assist
 you
 today
?

So, what I mean is, we should have a switch to control the printing of function call parameters, so that we can distinguish which ones are the streaming outputs of return_direct.

tmc / langchaingo

openai: add toolcalls support when streaming #763

PR Checklist

PR Checklist