Async bug? - Githubissues

ridwanarf25 commented 1 year ago

This is the error prompt from simple-proxy-for-tavern in the terminal

{ user: 'You', assistant: 'Saga' }
final prompt tokens = 1797
{ stop_sequence: [ '\n##', '\n*You', '\nYou:' ] }
TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11457:11)
    at async koboldGenerate (file:///media/sda2/.../SillyTavern/simple-proxy-for-tavern/index.mjs:530:16)
    at async getChatCompletions (file:///media/sda2/.../SillyTavern/simple-proxy-for-tavern/index.mjs:729:5)
    at async Server.<anonymous> (file:///media/sda2/...SillyTavern/simple-proxy-for-tavern/index.mjs:774:7)
POST /v1/chat/completions
COMPLETIONS {

This is how koboldcpp terminal output

127.0.0.1 - - [30/Apr/2023 00:31:44] "POST /api/v1/generate HTTP/1.1" 200 -
Generate: The response could not be sent, maybe connection was terminated?

This is how SillyTavern terminal output

  ],
  model: 'gpt-3.5-turbo-0301',
  temperature: 0.9,
  frequency_penalty: 0.7,
  presence_penalty: 0.7,
  max_tokens: 355,
  stream: false,
  reverse_proxy: 'http://127.0.0.1:29172/v1'
}
{ error: 'fetch failed' }
{

Sometime works flawlessly but sometimes it can't get the output from koboldcpp latest release, idk what triggered it. I'm using vicuna 7b 1.1 and firefox

anon998 commented 1 year ago

Does this still happens if you switch to the LTS version of Node.js? That or try to force it into using the node-fetch package by commenting line 17 and 21 here: https://github.com/anon998/simple-proxy-for-tavern/blob/95837e1f60e288ef8b8f8b07910a5d5589b18f2f/index.mjs#L17

anon998 commented 1 year ago

screenshot I think this is fixed now, so I'm going to close it. It now uses the node-fetch package and the readme now says to install the latest LTS version of Node.js.

anon998 / simple-proxy-for-tavern

Async bug? #2