How to implement nested function calling using VoicePipelineAgent?

[MartinKovachev]

Hello LiveKit team @davidzhao @dsa,

We’re working on a project that utilizes LiveKit and have encountered a couple of challenges that we were unable to resolve with the existing documentation and examples. Specifically, we’d appreciate your guidance on the following:

1. Nested Function Calling: We’re looking for an example or best practices on handling nested function calls within LiveKit. We’d like to understand how to manage the flow and error handling when one function needs to call another, especially with a focus on minimizing latency and optimizing performance.

2. Dynamic Function Addition in FunctionContext: Additionally, we’d like to see an example of dynamically adding functions in the FunctionContext class. This functionality is crucial for our application, but we’re unsure of the correct approach to implement it.

Thank you very much for your assistance and for all your hard work on LiveKit!

[davidzhao]

Hey @MartinKovachev, for 1. we do support nested function calls, you can increase max_nested_fnc_calls a higher value.

we left default low because most LLMs other than OpenAI do not handle it properly.

for 2. we are working on improvements in this API to make this simpler. this will be in 0.12.x, to be released later this month

[MartinKovachev]

Thank you @davidzhao