ztjhz / BetterChatGPT

An amazing UI for OpenAI's ChatGPT (Website + Windows + MacOS + Linux)
http://bettergpt.chat/
Creative Commons Zero v1.0 Universal
7.8k stars 2.64k forks source link

handle `\r` gracefully as chunk separator in completion events #537

Open khimaros opened 4 months ago

khimaros commented 4 months ago

currently llama-cpp-python provides an OpenAI compatible API, but it joins multiple chunks into a single completion event with \r\r instead of \n, which causes the following exception:

VM190:3 Uncaught SyntaxError: Unexpected non-whitespace character after JSON at position 223
    at JSON.parse (<anonymous>)
    at eval (eval at <anonymous> (index-3e43d292.js:294:74), <anonymous>:1:6)
    at index-3e43d292.js:294:74
    at Array.map (<anonymous>)
    at L6e (index-3e43d292.js:293:20)
    at handleSubmit (index-3e43d292.js:305:824)

i am also following up with llama-cpp-python to see if i can upstream a fix, but it seems worthwhile for BetterChatGPT to be resilient to this sort of input.

khimaros commented 3 months ago

happy to share that llama-cpp-python >= 0.2.38 uses newline delimited events. it would still be nice to be more resilient to other delimiters.

note however that https://github.com/ztjhz/BetterChatGPT/issues/538 still requires a workaround in llama-cpp-python to disable the ping events, so it does not reliably work out of the box.