search

simonw / llm-llama-cpp

LLM plugin for running models using llama.cpp
Apache License 2.0
136 stars 19 forks source link

Indicate streaming support. Fixes #11 #12

Closed michitux closed 1 year ago

michitux commented 1 year ago

I copied this from https://github.com/simonw/llm-mlc/blob/b05eec9ba008e700ecc422c69e28395fa8e146fd/llm_mlc.py#L214 and from a short test this seems to work.

simonw commented 1 year ago

Wow, thanks for catching this!