Enable streaming for chains and structured output

Maximilian-Winter / llama-cpp-agent

The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM models, execute structured function calls and get structured output. Works also with models not fine-tuned to JSON output and function calls.

Other

472 stars 42 forks source link

Enable streaming for chains and structured output #67

Closed pabl-o-ce closed 3 months ago

pabl-o-ce commented 4 months ago

Possible to give some guide how enable streaming in chains and structured output.

Maximilian-Winter commented 3 months ago

I have tried it, but with chain it is not possible at the moment.