search

karthink / gptel

A simple LLM client for Emacs
GNU General Public License v3.0
1.04k stars 111 forks source link

(Gemini) Empty Response OR Error in process sentinel #253

Closed karthink closed 3 months ago

karthink commented 3 months ago 
          I deleted and reloaded gptel. I still had (setq gptel-stream nil) in my init, so I got the sentinel error I described earlier.

After I changed that, I got Gemini error (HTTP/2 503): Malformed JSON in response.

This was apparently because the model was overloaded, but maybe the error should be different?

I tried it again and got the empty response message. But in fact, the response in gptel-log did not look empty.

Here it is:

{
  "gptel": "response headers",
  "timestamp": "2024-03-15 00:32:47"
}
"HTTP/2 200 \r\ncontent-type: application/json; charset=UTF-8\r\nvary: Origin\r\nvary: X-Origin\r\nvary: Referer\r\ncontent-encoding: gzip\r\ndate: Fri, 15 Mar 2024 04:32:40 GMT\r\nserver: scaffolding on HTTPServer2\r\ncache-control: private\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nx-content-type-options: nosniff\r\nserver-timing: gfet4t7; dur=773\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r"
{
  "gptel": "response body",
  "timestamp": "2024-03-15 00:32:47"
}
[
  {
    "candidates": [
      {
        "content": {
          "parts": [
            {
              "text": "Canine companion; domesticated carnivoran"
            }
          ],
          "role": "model"
        },
        "finishReason": "STOP",
        "index": 0,
        "safetyRatings": [
          {
            "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
            "probability": "NEGLIGIBLE"
          },
          {
            "category": "HARM_CATEGORY_HATE_SPEECH",
            "probability": "NEGLIGIBLE"
          },
          {
            "category": "HARM_CATEGORY_HARASSMENT",
            "probability": "NEGLIGIBLE"
          },
          {
            "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
            "probability": "NEGLIGIBLE"
          }
        ]
      }
    ],
    "promptFeedback": {
      "safetyRatings": [
        {
          "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_HATE_SPEECH",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_HARASSMENT",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
          "probability": "NEGLIGIBLE"
        }
      ]
    }
  }
]

Just FYI, here's the response when I got the 503 message:

{
  "gptel": "response headers",
  "timestamp": "2024-03-15 00:27:31"
}
"HTTP/2 503 \r\nvary: Origin\r\nvary: X-Origin\r\nvary: Referer\r\ncontent-type: application/json; charset=UTF-8\r\ncontent-encoding: gzip\r\ndate: Fri, 15 Mar 2024 04:27:24 GMT\r\nserver: scaffolding on HTTPServer2\r\ncache-control: private\r\ncontent-length: 135\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nx-content-type-options: nosniff\r\nserver-timing: gfet4t7; dur=177\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r"
{
  "gptel": "response body",
  "timestamp": "2024-03-15 00:27:31"
}
[
  {
    "error": {
      "code": 503,
      "message": "The model is overloaded. Please try again later.",
      "status": "UNAVAILABLE"
    }
  }
]

And thank you for the package!

Originally posted by @mmwilbert in https://github.com/karthink/gptel/issues/251#issuecomment-1998925686

karthink commented 3 months ago

@mmwilbert Does (json-available-p) return t or nil for you?

mmwilbert commented 3 months ago

It returns nil. I recompiled the emacs on that platform to include json and now it returns t.

However, after that recompile, using gptel-send still gives me "Empty response", although there is clearly a response in the gptel-log.

As far as I can see, the log is basically the same: { "gptel": "request headers", "timestamp": "2024-04-02 17:31:22" } { "Content-Type": "application/json", "Authorization": "Bearer nonono" } { "gptel": "request body", "timestamp": "2024-04-02 17:31:22" } { "model": "gpt-3.5-turbo", "messages": [ { "role": "system", "content": "You are a large language model living in Emacs and a helpful assistant. Respond concisely." }, { "role": "user", "content": "what is an aardwolf?" } ], "stream": true, "temperature": 1.0 } {"gptel": "request Curl command", "timestamp": "2024-04-02 17:31:22"} curl \ --disable \ --location \ --silent \ --compressed \ -XPOST \ -y300 \ -Y1 \ -D- \ -w(ecedcb0c31ab35c5ce1781812d6f1cf4\ .\ \%{size_header}) \ -d{\"model\"\:\"gpt-3.5-turbo\"\,\"messages\"\:[{\"role\"\:\"system\"\,\"content\"\:\"You\ are\ a\ large\ language\ model\ living\ in\ Emacs\ and\ a\ helpful\ assistant.\ Respond\ concisely.\"}\,{\"role\"\:\"user\"\,\"content\"\:\"what\ is\ an\ aardwolf\?\"}]\,\"stream\"\:true\,\"temperature\"\:1.0} \ -HContent-Type\:\ application/json \ -HAuthorization\:\ Bearer\ nonono \ https\://api.openai.com/v1/chat/completions { "gptel": "response headers", "timestamp": "2024-04-02 17:31:24" } "HTTP/2 200 \r\ndate: Tue, 02 Apr 2024 21:31:22 GMT\r\ncontent-type: text/event-stream\r\naccess-control-allow-origin: *\r\ncache-control: no-cache, must-revalidate\r\nopenai-model: gpt-3.5-turbo-0125\r\nopenai-organization: user-tebtekbenm9hxoxd0m1zysee\r\nopenai-processing-ms: 1314\r\nopenai-version: 2020-10-01\r\nstrict-transport-security: max-age=15724800; includeSubDomains\r\nx-ratelimit-limit-requests: 10000\r\nx-ratelimit-limit-tokens: 60000\r\nx-ratelimit-remaining-requests: 9999\r\nx-ratelimit-remaining-tokens: 59954\r\nx-ratelimit-reset-requests: 8.64s\r\nx-ratelimit-reset-tokens: 46ms\r\nx-request-id: req_51d0aeacb2320d5d4c1223f87dabdc54\r\ncf-cache-status: DYNAMIC\r\nset-cookie: __cf_bm=iIsOzfCY0OPLE8dlH.yQBPXa1MHOlcVHoG5WM1DkJUk-1712093482-1.0.1.1-BdyIXetA.K_FrUb1iT70o0PXESmB0Ud6LLPQ0N.j51.PUQWULFcnST9BDR1vvQ5BKPwwHBBeL329VP_knpSlQg; path=/; expires=Tue, 02-Apr-24 22:01:22 GMT; domain=.api.openai.com; HttpOnly; Secure; SameSite=None\r\nset-cookie: _cfuvid=vBEky2rY1OX8_WWATYh_BG_46DEcA19X4R0_QQNYNBo-1712093482598-0.0.1.1-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None\r\nserver: cloudflare\r\ncf-ray: 86e3ece0ed688ff1-BOS\r\nalt-svc: h3=\":443\"; ma=86400\r\n\r" { "gptel": "response body", "timestamp": "2024-04-02 17:31:24" } data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":"An"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" a"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":"ard"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":"wolf"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" is"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" a"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" small"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" insect"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":"-e"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":"ating"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" mamm"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":"al"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" native"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" to"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" eastern"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" and"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" southern"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" Africa"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" It"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" is"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" part"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" of"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" the"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" hy"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":"ena"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" family"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" but"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" mainly"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" feeds"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" on"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":" ter"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":"mites"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-99g0wOo15Es9ipgIkcUYD17Vy6UBh","object":"chat.completion.chunk","created":1712093482,"model":"gpt-3.5-turbo-0125","system_fingerprint":"fp_b28b39ffa8","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}]}

data: [DONE]

On Tue, Apr 2, 2024, at 3:10 PM, karthink wrote:

@mmwilbert https://github.com/mmwilbert Does (json-available-p) return t or nil for you?

— Reply to this email directly, view it on GitHub https://github.com/karthink/gptel/issues/253#issuecomment-2032862838, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH5KWDGC4N24QZXEXQVUKWDY3L7C5AVCNFSM6AAAAABEYJPLEOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZSHA3DEOBTHA. You are receiving this because you were mentioned.Message ID: @.***>

karthink commented 3 months ago

Please rebuild/reinstall gptel, I fixed a very similar problem (#260) recently.

Rebuilding Emacs was not required -- you don't need libjansson support to use gptel, the lisp-based json parsing is fast enough. I only asked if json-available-p returns nil since this was the case where the bug was manifesting.

mmwilbert commented 3 months ago

That fixed it. It is working on 29.2 connecting to ChatGPT. I also tried it with Claude and Gemini and they all work now.

Thank you very much!

Matt

On Tue, Apr 2, 2024, at 6:10 PM, karthink wrote:

Please rebuild/reinstall gptel, I fixed a very similar problem (#260 https://github.com/karthink/gptel/issues/260) recently.

Rebuilding Emacs was not required -- you don't need libjansson support to use gptel, the lisp-based json parsing is fast enough. I only asked if json-available-p returns nil since this was the case where the bug was manifesting.

— Reply to this email directly, view it on GitHub https://github.com/karthink/gptel/issues/253#issuecomment-2033188181, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH5KWDCOUEQNWKLVK2MQ7MLY3MUDZAVCNFSM6AAAAABEYJPLEOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZTGE4DQMJYGE. You are receiving this because you were mentioned.Message ID: @.***>