Error message using Kobold

[TehNomad]

I'm using KoboldAI 4-bit branch as my backend and I'm getting the following error message when I submit a prompt in SillyTavern to the proxy:

TypeError: Cannot set properties of undefined (setting 'abortPreviousRequest')
    at koboldGenerateStreamUI2 (file://***/simple-proxy-for-tavern/src/kobold-stream.mjs:246:30)
    at koboldGenerate (file://***/simple-proxy-for-tavern/src/backends.mjs:87:11)
    at getChatCompletions (file://***/simple-proxy-for-tavern/src/index.mjs:490:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Server.<anonymous> (file://***/simple-proxy-for-tavern/src/index.mjs:561:9)

[anon998]

I didn't realize that was broken. Try editing this line to add abort at the end: https://github.com/anon998/simple-proxy-for-tavern/blob/e1a0ef30d09aee28947d8e0cd4057b4c5becdca5/src/backends.mjs#L87 Like this:

await koboldGenerateStreamUI2(req, res, params, config, abort);

I can't test it right now. If it doesn't work with that you will have to disable streaming in Tavern until I look at it.

[TehNomad]

Yes, that fixed that error. Now it seems like it's getting stuck on "loading options." I'll investigate a bit on my own, but do you have any suggestions on how to debug this?

EDIT: Never mind, I fixed this by disabling streaming (Kobold 4-bit with exllama doesn't support this I think).

[anon998]

Kobold 4-bit with exllama doesn't support this I think

I tried it and it worked. It's just that you have to keep Kobold's window open for it to work. I need to investigate that more, I think Kobold's UI sends one more thing that's missing and that's why it works when it's open.

[TehNomad]

Yep, keeping the window open also works for me. Thanks for looking into both of those issues.