LLM context aggregator

[daedsidog]

This adds the contexter I've been working on for my own personal use to GPTel. As of now, this pull request is not complete because I'm unsure what's the best path to integrate it, and also how it will fit with the new UI refactor. I synced up with the master branch of GPTel, and just added that in, but my transient menu seems to be vastly different from the previews I've seen.

Because this started as part of my config & grew organically, there are still some features that are part of my config and not in GPTel. Those parts are added to the end of this pull request in the form of a snippet from my config.

In a nutshell, I personally find this to be an incredibly useful programming tool for productivity, more so than any other AI programming tool I've tried. It pampered me to the point where I don't think I can go back to programming without it.

Here is a short guide how to use it with my config (this will change the more things I'll offload to GPTel):

1. Select various regions in buffers and add them to the context buffer with C-c g p. You can press c to remove selected context snippets from the context buffer directly, or C-c g p again to remove context snippets from the code buffers.

2. Write an instruction comment in code, then select it and use C-c g r to send it to GPTel.

With Lisp, the contexter automatically collapses s-expressions in areas that they are not required in, such as functions with docstrings, thus saving tokens:

Here is the part of my config file which sets all this up:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; -------------------------------- GPTEL --------------------------------- ;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(use-package gptel-contexter
  :commands (gptel-context-string
             gptel-context-substring
             gptel-context-in-region
             gptel-pop-or-push-context
             gptel-remove-context))

(defun my/code-refactoring-prompt-string (code-regions)
  "Create a prompt for an LLM to refactor CODE-REGIONS in the current buffer."
  (let* ((context (gptel-context-string))
         (have-context (not (zerop (length context)))))
    (concat "I need you to refactor and/or write code. Write carefully with\
 great respect to existing syntax, styling, & conventions, this also includes\
 column width."
            (when have-context
              (concat "Listed below is context that may help you. Use it as a\
 read-only reference. You are strictly prohibited from changing it in any\
 way, even if asked directly."
                      "\n\n" context))
            (if have-context
                "\n\nNow, given the above context, do as instructed by the\
 INSTRUCTION comments in the following source:"
              "\n\nDo as instructed by the INSTRUCTION comments in the following\
 source:")
            "\n\n"
            (gptel-context-substring (current-buffer) code-regions)
            "\n\n"
            "When writing code, do not delete existing comments. Do not fix\
 what you may think are bugs, unless explicitly instructed to. Do not refactor\
 things you were not instructed to. You are to remove instruction comments\
 after they are completed.

You have more creative freedom when you are generating code, but you must still\
 follow your instructions closely.

As opposed to the context, the source code you have been given has been erased\
 from its original location. Therefore, you are expected to return it, fully.\
 Of course, you are not to return the ellipses or line numbers. Remember, you\
 may have been given partial code, so be mindful and return it in the same way\
 you found it. Return **ALL** of the modified target code, leave nothing behind.

Return only code, nothing else. When documenting your work, make no reference\
 to the instructions, so that an outside reader will think the code was written\
 in a natural fashion.")))

(defun my/predicated-prog-lang (prog-lang-major-mode)
  "Derive the programming language name from the PROG-LANG-MAJOR-MODE.
Adds a predicate before the name."
  (pcase prog-lang-major-mode
    ('emacs-lisp-mode "an Emacs Lisp")
    ('js-mode "a JavaScript")
    ('c-mode "a C")
    ('c++-mode "a C++")
    ('lisp-mode "a Common Lisp")
    ('twee-mode "a Twee")
    ('web-mode "a Web")
    (_ (concat "a " (substring (symbol-name prog-lang-major-mode) nil -5)))))

(defun my/refactoring-message ()
    "Set a generic refactor/rewrite message for the buffer."
    (if (derived-mode-p 'prog-mode)
        (concat (format "You are %s programmer. Write legible, elegant, and\
 very terse code based on instructions. Do not stray from instructions."
                        (my/predicated-prog-lang major-mode)))
      (format "You are a prose editor. Rewrite the following text to be more\
 professional.")))

(defun my/clean-up-llm-chat-response (beg end)
  (save-excursion
    (goto-char beg)
    (org-latex-preview)
    (my/resize-org-latex-overlays)
    (my/adjust-org-latex-overlay-padding)
    (pulse-momentary-highlight-region beg end)))

(use-package gptel
  :commands (gptel-clean-up-llm-code))

(defun my/clean-up-llm-response (beg end)
  (if (bound-and-true-p gptel-mode)
      (my/clean-up-llm-chat-response beg end)
    (gptel-clean-up-llm-code (current-buffer) beg end)))

;; OpenAI ChatGPT API interface
(use-package gptel-curl
  :commands (gptel gptel-menu gptel-abort gptel-clean-up-llm-code)
  :config
  (require 'gptel-transient)
  (customize-set-variable 'gptel-directives
                          (let ((new-directives gptel-directives))
                            (setf (cdr (assoc 'default new-directives))
                                  "You are a large language model living in\
 Emacs and a helpful assistant. Respond as concisely as possible.")
                            new-directives))
  (customize-set-variable 'gptel-default-mode 'org-mode)
  (customize-set-variable
   'gptel-prompt-prefix-alist
   (cons (cons 'org-mode "* Query\n\n")
         (assq-delete-all 'org-mode gptel-prompt-prefix-alist)))
  (customize-set-variable
   'gptel-response-prefix-alist
   (cons (cons 'org-mode "** Response\n\n")
         (assq-delete-all 'org-mode gptel-response-prefix-alist)))
  (customize-set-variable 'gptel-model "gpt-4-0125-preview")
  (add-hook 'gptel-mode-hook
            (lambda () (interactive)
              (display-time-mode)
              (setq gptel--system-message
                    (alist-get 'default gptel-directives))))
  (add-hook 'gptel-post-response-functions #'my/clean-up-llm-response)
  :bind (("C-c g c" . #'gptel)
         ("C-c g m" . #'gptel-menu)
         ("C-c g p" . #'gptel-pop-or-push-context)
         ("C-c g r" . #'my/refactor-using-llm-with-context)
         ("C-c g a" . #'gptel-abort)
         (:map gptel-mode-map
               ;; Insert a prefix when deleting the entire buffer in
               ;; `gptel-mode'.
               (("C-<tab> DEL" . (lambda ()
                            (interactive)
                            (my/delete-entire-buffer)
                            (catch 'finish
                              (dolist (it gptel-prompt-prefix-alist)
                                (let ((mode (car it))
                                      (prefix (cdr it)))
                                  (when (eq major-mode mode)
                                    (insert prefix)
                                    (goto-char (point-max))
                                    (throw 'finish nil)))))))))))

(cl-defun my/refactor-using-llm-with-context ()
  (interactive)
  (unless (use-region-p)
    (cl-return-from my/refactor-using-llm-with-context))
  (let* ((start (region-beginning))
         (end (region-end))
         (context (gptel-context-in-region (current-buffer) start end))
         (prompt)
         (region-substring (buffer-substring-no-properties start end)))
    (when context
      (gptel-remove-context context t))
    ;; Check for the presence of instruction in region.
    (when (not (string-match-p "INSTRUCTION:" region-substring))
      (error "No instruction found within region"))
    (setq prompt (my/code-refactoring-prompt-string (list (list start end))))
    (deactivate-mark)
    (delete-region start end)
    (set-mark (point))
    (insert prompt)
    (activate-mark)
    (gptel-menu)
    ;; Set the system message through the transient menu and keybord
    ;; macros, which is just a temporary solution.
    (execute-kbd-macro (kbd "ss"))
    (delete-region (point) (point-max))
    (insert (my/refactoring-message))
    (execute-kbd-macro (kbd "C-c C-c"))
    (execute-kbd-macro (kbd "rR"))))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; ----------------------------- END OF GPTEL ----------------------------- ;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

As can be plainly seen from the config file, the mechanism to actually send this to GPTel is rather primitive. It uses keyboard macros to navigate the transient menu, and then just dumps the prompt into the buffer, and uses the transient refactor menu to send it to GPTel.

[karthink]

They are currently not customizable, but are only used in gptel-context-wrap-function through gptel-context--string. If the user has their own custom wrapping function, they are never used again. If the user wishes only to have access to the context string and nothing more, he is limited to the existing gptel-context--string.

I guess what I am trying to say is that the current way these 3 functions are written do little do suggest that they are just "default suggestions", especially since they use Markdown in the generated strings.

Yeah, I'm not inclined to make the string creation function separately customizable. Writing a gptel-context-wrap-function is only one additional step on top of creating the strings, so string creation should be covered. And if they really want to modify only the string creation part they can advise gptel-context--string.

[karthink]

Anywho, everything else is on point. Cheers!

Sounds good. I'll update the README squash these commits into logical ones (as best I can), and merge it into feature-context, then master this weekend. Hopefully today if I can find more free time.

[karthink]

Oh, one last outstanding issue is the -i Include (with system message) option in the transient menu. Should I move it into the Request Parameters section to reinforce that it works like the other settings (i.e. persistent and possibly buffer-local)?

[daedsidog]

Oh, one last outstanding issue is the -i Include (with system message) option in the transient menu. Should I move it into the Request Parameters section to reinforce that it works like the other settings (i.e. persistent and possibly buffer-local)?

I think it would be better, despite the crowding.

Did you by any chance do something with a6031af89685f82714397e7b6e1ab10e820ff136 and b7956e427fb09d9eae908a42f50ef6a4e29410c3? I noticed they have been superseded. If so, you didn't like the way it looked like? The whitespaces between the separators & the superimposed overlays? The superimposed overlays made it easier to see how highlighted overlays were marked.

EDIT: You didn't change the superimposed overlays, but removing the final newline makes single-line contexts be hidden under the highlighting overlay:

![Uploading image.png…]()

EDIT 2: Actually, even if I add the last newline, it doesn't show the superimposed overlay like in the first context. Strange, was sure it used to do that even for single-line contexts.

![Uploading image.png…]()

[karthink]

Two different things happened here.

https://github.com/karthink/gptel/commit/a6031af89685f82714397e7b6e1ab10e820ff136: This was overwritten by accident, I'll add it back.

https://github.com/karthink/gptel/commit/b7956e427fb09d9eae908a42f50ef6a4e29410c3: I only changed the code to not use a first local var. Give me a minute to test single line contexts, I tested it but did not see what you see in the above image 1.

[karthink]

I can't reproduce your image with single-line contexts:

With highlighting:

[daedsidog]

I can't reproduce your image with single-line contexts:

How are you adding your context?

In the above, the top line is me adding a line + its newline, and for the rest, I just add their lines with by marking the start of the line and using line-end, then calling gptel-context-add on the selection. This coincides with adding the region with the transient menu.

You probably add the entire line by doing it similarly, only you just go to the start of the next line.

[karthink]

Also it looks like you overwrote your own change to gptel-context-previous in the consecutive commits c0303310099b4da826e21a3b09e4954b8743e63e and b7956e427fb09d9eae908a42f50ef6a4e29410c3.

[karthink]

You probably add the entire line by doing it similarly, only you just go to the start of the next line.

No, here's the difference I see between picking a part of a line and a "whole line" (line + following newline).

[daedsidog]

No, here's the difference I see between picking a part of a line and a "whole line" (line + following newline).

Do you have an actual newline between the separator and the text? Could just be how my Emacs renders that separator. In a terminal version for example it appears fine:

But there's really no newline there.

I'll try with an empty config. EDIT: Getting transient-setup: Wrong type argument: (or eieio-object cl-structure-object oclosure), "" when invoking gptel-menu with emacs -q. So much for that...

[karthink]

Do you have an actual newline between the separator and the text? Could just be how my Emacs renders that separator. In a terminal version for example it appears fine:

There is one newline between the text and the following separator:

Screencast_20240622_151316.webm

[karthink]

Also it looks like you overwrote your own change to gptel-context-previous in the consecutive commits c030331 and b7956e4.

Hope you saw this message by the way. I never saw these changes in my working directory as a result. You can add it back.

[daedsidog]

There is one newline between the text and the following separator:

~I don't think so~ I think there should be two. In the first context, the line directly after the context text contains the separator, and the next line is the newline between the next text and the separator. In my Emacs the separator is its own line, just very shrunk.

Not sure what accounts for this difference, though.

[karthink]

Not sure what accounts for this difference, though.

Okay, I found the problem: separator lines are implemented differently in gtk and non-gtk Emacsen. This is probably also platform dependent. I added a second newline to be safe. The spacing in terminal and non-gtk Emacs will look weird, but we've already spent more time worrying about this than anyone should have to worry about blank lines.

[daedsidog]

Not sure what accounts for this difference, though.

Okay, I found the problem: separator lines are implemented differently in gtk and non-gtk Emacsen. This is probably also platform dependent. I added a second newline to be safe.

Also, when selecting an entire line, I get this:

I.e., single-line context highlight throughout the buffer. This looks much better than the first version.

(Note that the image above does not account for the latest commit. It's just an observation.)

[daedsidog]

(Note that the image above does not account for the latest commit. It's just an observation.)

I suppose I somehow convinced myself that this behavior existed at one point, but it probably didn't. Implementing this would require to have the deletion overlay end one point after the context overlay, and the complication it would introduce to navigation makes it not really worth it in my opinion.

[daedsidog]

I suppose I somehow convinced myself that this behavior existed at one point, but it probably didn't. Implementing this would require to have the deletion overlay end one point after the context overlay, and the complication it would introduce to navigation makes it not really worth it in my opinion.

Actually, whether or not this is correct no longer matters, as I made it work like that with minimal changes to gptel-context--buffer-setup:

Looking sharp!

[karthink]

Planning to merge in ~2 hours, in case you want to make any more small changes.

(Or I can wait for a few days if you want to address something non-trivial about the PR)

[daedsidog]

Planning to merge in ~2 hours, in case you want to make any more small changes.

(Or I can wait for a few days if you want to address something non-trivial about the PR)

Looks great to me. I think you exhausted every feature!

I would check a clean config to see if we (I) didn't introduce some kind of bug to the transient menu, though:

I'll try with an empty config. EDIT: Getting transient-setup: Wrong type argument: (or eieio-object cl-structure-object oclosure), "" when invoking gptel-menu with emacs -q. So much for that...

If you can't reproduce it on your end, I'll account it for a bad, unsupported setup of the package on my account.

[karthink]

If you can't reproduce it on your end, I'll account it for a bad, unsupported setup of the package on my account.

I tested it with emacs -q + package-install-file yesterday, mainly to test the autoloading of gptel-context.el, and everything was fine.

I'll test it again before merging.

[daedsidog]

🥳

[karthink]

Merged into feature-context, and now master. Fingers crossed!

Thanks for the PR @daedsidog! Took a while but we got there.