Claude error

[kjnez]

Hi there,

Thanks for creating and maintaining this brilliant Emacs package!

When using Claude Sonnet 3.5 API with org-mode, I often run into the following error:

Querying Claude... Claude error: (HTTP/2 400) messages.4: all messages must have non-empty content except for the optional final assistant message

I never had the same problem when using ChatGPT.

[karthink]

As the message says, Claude requires the first message in a chain to be a user message. What is the text you are sending Claude?

[kjnez]

Just normal messages. For example,

Question 1 text goes here Claude's answer here Question 2 text goes here Claude's answer here *** Question 3 text goes here Error occurred when hitting C-c Enter

[karthink]

Okay, run (setq gptel-expert-commands t), then from gptel's transient menu, use the dry run option to inspect the query as Lisp or JSON. You can paste the result of that here.

[kjnez]

Here's the query as Lisp:

(:model "claude-3-5-sonnet-20240620" :system "The user is a programmer with very limited time. You treat their time as precious. You do not repeat obvious things, including their query. You are as concise as possible in responses. You never apologize for confusions. Don't explain code snippets." :stream t :max_tokens 1024 :messages
    [(:role "user" :content "Does Elisp support async programming?")
     (:role "assistant" :content "Yes, Emacs Lisp supports asynchronous programming through several mechanisms:

1. =run-at-time= and =run-with-timer= for delayed execution
2. =run-with-idle-timer= for running code when Emacs is idle
3. =make-process= and =start-process= for asynchronous external processes
4. =async.el= library for more advanced async capabilities
5. =promise.el= for promise-based asynchronous programming

These allow non-blocking operations and background tasks in Emacs.")
     (:role "user" :content "Show me how promise.el works")
     (:role "assistant" :content "Here's a basic example using promise.el:

#+begin_src elisp")
     (:role "user" :content "")
     (:role "assistant" :content "(require 'promise)")
     (:role "user" :content "")
     (:role "assistant" :content "(defun async-operation ()")
     (:role "user" :content "")
     (:role "assistant" :content "(promise-new (lambda (resolve reject)")
     (:role "user" :content "")
     (:role "assistant" :content "")
     (:role "user" :content "")
     (:role "assistant" :content "(run-with-timer 2 nil (lambda () (funcall resolve \"Result\"))))))")
     (:role "user" :content "")
     (:role "assistant" :content "(promise-chain (async-operation)")
     (:role "user" :content "")
     (:role "assistant" :content "")
     (:role "user" :content "")
     (:role "assistant" :content "(then (lambda (result) (message \"Got: %s\" result)))")
     (:role "user" :content "")
     (:role "assistant" :content "(catch (lambda (error) (message \"Error: %s\" error))))
#+end_src

This creates a promise that resolves after 2 seconds, then chains operations to handle the result or errors. The promise-chain macro simplifies promise composition.")
     (:role "user" :content "Show me how async.el works")]
    :temperature 1.0)

Now I know what triggers this problem. When the LLM is generating code in org-mode, for some reason the syntax is not highlighted. So sometimes I'd go to the beginning of the generated code (in the above case, after #+begin_src elisp) and create a new line then delete it. The syntax will be highlighted and everything looks normal. But from the above inspection, it seems that this action injected empty user input at each line of the code.

Here's the whole output

*** Does Elisp support async programming?

Yes, Emacs Lisp supports asynchronous programming through several mechanisms:

1. =run-at-time= and =run-with-timer= for delayed execution
2. =run-with-idle-timer= for running code when Emacs is idle
3. =make-process= and =start-process= for asynchronous external processes
4. =async.el= library for more advanced async capabilities
5. =promise.el= for promise-based asynchronous programming

These allow non-blocking operations and background tasks in Emacs.

*** Show me how promise.el works

Here's a basic example using promise.el:

#+begin_src elisp
  (require 'promise)

  (defun async-operation ()
    (promise-new (lambda (resolve reject)
      (run-with-timer 2 nil (lambda () (funcall resolve "Result"))))))

  (promise-chain (async-operation)
    (then (lambda (result) (message "Got: %s" result)))
    (catch (lambda (error) (message "Error: %s" error))))
#+end_src

This creates a promise that resolves after 2 seconds, then chains operations to handle the result or errors. The promise-chain macro simplifies promise composition.

*** Show me how async.el works

[kjnez]

By the way, this is the function that I used to create a new line:

(defun newline-and-indent-without-breaking-line ()
  (interactive)
  (end-of-line)
  (newline-and-indent))

[karthink]

Yeah, editing llm responses is an issue, see #321. Fixing this is on the docket, but I'm not working on it right now. You can avoid this error by not producing a blank user prompt as the first message.

[kjnez]

Got it. What about the syntax highlighting issue? When using Chatgpt, the syntax is highlighted, but with Claude, sometimes it is not. The behavior is inconsistent.

[karthink]

Got it. What about the syntax highlighting issue? When using Chatgpt, the syntax is highlighted, but with Claude, sometimes it is not. The behavior is inconsistent.

What's the syntax highlighting issue? Please create a new issue if required.

[kjnez]

As I mentioned in my previous comment:

Now I know what triggers this problem. When the LLM is generating code in org-mode, for some reason the syntax is not highlighted. So sometimes I'd go to the beginning of the generated code (in the above case, after #+begin_src elisp) and create a new line then delete it. The syntax will be highlighted and everything looks normal. But from the above inspection, it seems that this action injected empty user input at each line of the code.

The syntax seems not always to be highlighted with Claude's generated code in org-mode. That's why I started to edit the code snippet in order for it to be syntax highlighted.

[karthink]

As I mentioned in my previous comment:

Now I know what triggers this problem. When the LLM is generating code in org-mode, for some reason the syntax is not highlighted. So sometimes I'd go to the beginning of the generated code (in the above case, after #+begin_src elisp) and create a new line then delete it. The syntax will be highlighted and everything looks normal. But from the above inspection, it seems that this action injected empty user input at each line of the code.

The syntax seems not always to be highlighted with Claude's generated code in org-mode. That's why I started to edit the code snippet in order for it to be syntax highlighted.

The "begin_src elisp" and "end_src" blocks are generated correctly, but syntax highlighting doesn't happen? If this is the case this is a problem with jit-lock, not gptel. Try running (font-lock-ensure) instead of modifying the buffer. If that works, you can add it to the post-response hook: (add-hook 'gptel-post-response-functions 'font-lock-ensure).

[kjnez]

Yes the "begin_src elisp" and "end_src" blocks are generated correctly, but there's no syntax highlighting. I tried (add-hook 'gptel-post-response-functions 'font-lock-ensure) but it doesn't seem to help.

Also, I don't have the same problem when using ChatGPT. Could this be specific to Anthropic?

[kjnez]

The absence of syntax highlighting is not quite consistent. Below is an example returned from Claude. Note that only the third source block is highlighted.

[nitinrathi]

Try ALT-X org-mode, this fixes the syntax highlighting for me.

[karthink]

@kjnez Again, this is a problem with how jit-lock is doing fontification, not gptel. Could you try running M-x font-lock-update (bound to C-x x f by default) and seeing if that fixes the problem?

I would not run M-x org-mode as @nitinrathi suggests, this is wasteful at best and can cause issues depending on how many things you've stuffed into org-mode-hook.

[kjnez]

M-x font-lock-update solves the problem. Thanks.

[karthink]

Very well, you can add something like this to the hook gptel-post-response-functions:

(defun my/gptel-font-lock-update (&rest _)
  (when font-lock-mode
    (font-lock-update)))

(add-hook 'gptel-post-response-functions #'my/gptel-font-lock-update)

[nitinrathi]

This is beautiful.

[axelknock]

Since the issue only presents in org mode:

(defun my/gptel-font-lock-update (&rest _)
    (when (and font-lock-mode (derived-mode-p 'org-mode))
      (font-lock-update)))
(add-hook 'gptel-post-response-functions #'my/gptel-font-lock-update)

[karthink]

I've pushed a fix, please test.