Open khimaros opened 9 months ago
happy to share that llama-cpp-python >= 0.2.38
uses newline delimited events. it would still be nice to be more resilient to other delimiters.
note however that https://github.com/ztjhz/BetterChatGPT/issues/538 still requires a workaround in llama-cpp-python to disable the ping events, so it does not reliably work out of the box.
currently llama-cpp-python provides an OpenAI compatible API, but it joins multiple chunks into a single completion event with
\r\r
instead of\n
, which causes the following exception:i am also following up with llama-cpp-python to see if i can upstream a fix, but it seems worthwhile for BetterChatGPT to be resilient to this sort of input.