sometimes Unexpected non-whitespace character ruines all the fun

[MartinAbilev]

SyntaxError: Unexpected non-whitespace character after JSON at position 104 (line 2 column 1) at JSON.parse () at Object.start (webpack-internal:///(middleware)/./utils/server/index.ts:46:45) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

this error I think is similar to one GPT API has too...

[GChristen]

I ran into this a little while back. This has to do with how the response, which can contain multiple completions in JSONL format (multiple JSON objects separated by new lines) is parsed. JSON.parse will fail on the JSONL format. You can modify the ReadableStream in utils/server/index.ts to handle 1 or more JSON objects and this should go away.

[ivanfioravanti]

@GChristen would you like doing a PR for this?

[marhar]

I am also getting this. I have a server based on the ollama APII spec. In non-streaming mode everything is OK, but in streaming mode I am also getting this error. I have tried with and without a NL between the response chunks. Talking to ollama directly works as expected.

Is there a convenient place in chat-bot ollama where I could print the offending line?

 ⨯ SyntaxError: Unexpected non-whitespace character after JSON at position 96 (line 1 column 97)
    at JSON.parse (<anonymous>)
    at Object.start (webpack-internal:///(middleware)/./utils/server/index.ts:46:45)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

My server log shows this being generated:

{"model":"foo-latest","created_at":"2024-02-24T12:41:04.537897","response":"B","done":false}
{"model":"foo-latest","created_at":"2024-02-24T12:41:04.542899","response":"ene","done":false}
{"model":"foo-latest","created_at":"2024-02-24T12:41:04.587043","response":"ath","done":false}