search

OvidijusParsiunas / deep-chat

Fully customizable AI chatbot component for your website
https://deepchat.dev
MIT License
1.27k stars 175 forks source link

Random "Error, please try again." errors while streaming #117

Closed tnunamak closed 4 months ago

tnunamak commented 4 months ago

image

I am using Deep Chat with a custom, OpenAI-compatible API as shown (styles omitted).

    <DeepChat
      initialMessages={[
        {"text": "Hello! How are you?", "role": "assistant"},
      ]}

      stream="true"
      directConnection={{
        stream: true,
        openAI: {
          chat: {
            model: 'TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf',
            system_prompt: `You count numbers, and never use words. NEVER use words. You only give short replies, like: 1, 2, 3, 4, 5.`,
            stop: ['[END]', '[INST]', '[/INST]'],
            temperature: 0,
            frequency_penalty: 0.7,
            presence_penalty: 0.7,
            top_p: 1,
            max_tokens: 350,
          },
          key: 'ignored'
        }
      }}
      request={{url: 'http://localhost:8181/v1/chat/completions' }}
    />

It seems that sometimes, extractResultData is called with a value for which choices is undefined, so result.choices[0].delta throws an error. I'm not sure why this is happening, because the event stream data sent by my backend looks correct and works with other clients.

In this case, the error appeared during streaming around the time that "1, 2, ___" was displayed. Logs from my backend are showing that there is nothing unusual about the event stream data being sent:

DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"role": "assistant"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " The"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " correct"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " answer"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " is"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " \""}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "three"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "\""}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " and"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " \""}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "six"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "\"."}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " So"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " the"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " sequence"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " would"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " be"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ":"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " \""}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "1"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ","}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " "}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "2"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ","}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " three"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ","}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " "}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "4"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ","}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " five"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ","}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " six"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ".\""}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "  "}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "\n"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "However"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " I"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " will"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " only"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " provide"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " one"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " number"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " at"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " a"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " time"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " to"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " follow"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " your"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " instruction"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " of"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " not"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " using"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " words"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "."}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " So"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " it"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " would"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " look"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " like"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " this"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ":"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " "}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "1"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ","}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " "}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "2"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ","}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " _"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "__,"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " "}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "4"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ","}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " _,"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " __"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "\n"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "An"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "swer"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ":"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " "}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "3"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": ","}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": " "}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {"content": "5"}, "finish_reason": null}]}
DEBUG:selfie.text_generation.generation:{"id": "chatcmpl-aa8ef51d-0415-463f-9b29-ea88655a6af1", "model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q5_K_M.gguf", "created": 1707231060, "object": "chat.completion.chunk", "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}]}

I'm happy to try debugging further, if you have any suggestions.

OvidijusParsiunas commented 4 months ago

Hi @tnunamak.

I have a quick change in the codebase to make choices optional and return {text: ''} if they are not present. You can access this code in deep-chat-dev and deep-chat-react-dev package versions 9.0.129. These packages work exactly the same as the normal ones except their names are different. If all will be good I will promote the changes to the core package later this week.

Small note, for anyone reading this thread - whilst mixing directConnection property with request can be a simple solution - it can provide some hurdles in the long run. If anything changes in the OpenAI API (or other directConnection service) in the future - we will have to make changes in how we call it, hence there is a chance that your target server may not be compatible to those changes. Also, when it comes to directConnection services that make multiple calls to different urls (e.g. OpenAI assistant), the request url will not be used as they have multiple hardcoded urls.

The best long-term direction to take is to ofcourse rely entirely on the request property and handle the required message processing on your server (example servers). I understand that this solution can be cumbersome, especially if you want to get going quick, hence I just wanted to bring it to your attention just-in-case.

Also for completeness, it is worth mentioning that you can also use the responseInterceptor to modify the incoming message bodies.

Hopefully this helps. Let me know if you have any questions and most importantly if the upgrade is sufficient. Thanks!

tnunamak commented 4 months ago

@OvidijusParsiunas thank you, it seems like that fixed it!

OvidijusParsiunas commented 4 months ago

The fix has now been released in Deep Chat version 1.4.11.