Include context by header instead of "number of responses"

[johanvts]

Thanks for an awesome tool.

It would be neat if the context could be controlled by header. So you can have a conversation like this (in org-mode):

* Topic A
** Question 1
Reponse
** Question 2
Response
* Topic B
** Question 1

And new questions in either topic fetch everything until the topic header as a context. I think this is currently done if I mark that region myself and have number of responses as nil? But I'm not sure and in any case it would be a nice default behavior (maybe also set the prompt default to * instead of ***).

[karthink]

I like the idea! It plays to the strengths of hierarchical markup. Implementing it (for both Org and Markdown) is straightforward, but the problem is going to be designing the UI around it. Consider:

* Topic A

** Question 1
Response 1

** Sub-topic B

*** Question 2
Response 2

*** Question 3
...

Should the context for Questions 2 and 3 be the Sub-topic B section, or Topic A? How do we indicate to the user what context is being sent?

[johanvts]

I think the included header level should be a variable and the default should be nil meaning include everything. And you simply set it to “*” or “**” etc. as for indicating what is sent I think maybe we can have the text flash like it does when outputting? But indication of the sent context is also currently an issue and I think separate from this idea.

[karthink]

And you simply set it to “*” or “**” etc

This is reasonable but not satisfying. The user might want to record a conversation at any indentation level in the buffer. For example:

* main heading
** sub-heading
*** Topic for chatgpt
Question

Answer

...

* main heading 2
** Another topic for chatgpt
*** Question

Answer

*** Question

Answer
...

This is not possible with a fixed indentation-level-context value like ** or ***. I think a better solution would be to use an Org heading tag (gptel, say, or chatgpt) for a heading that denotes a conversation. If no parent heading has such a tag, the contents of the buffer up to point are sent, which is the case right now.

So the above example becomes

* main heading
** sub-heading
*** Topic for chatgpt :gptel:
Question

Answer

...

* main heading 2
** Another topic for chatgpt :gptel:
*** Question

Answer

*** Question

Answer
...

It could also be an Org PROPERTY instead of a tag.

Something like this will work for org-mode but there's no bulit-in markdown equivalent.

[johanvts]

This is a great idea. Maybe the tag could be :gptel_new_context: (or something else that says this changes the context). Markdown and text mode doesn't have tags but I guess we could just use the same tag anyway?

So when the user hits gptel-send and this feature is active everything in the current buffer upwards until the first instance of :gptel_new_context: is included in the prompt.

[PalaceChan]

sounds cool, UI is tricky....how come "everything up until the first instance"? If it were something more around "everything with non-empty tag intersection with the header at point" then use of multiple tags could make a heading part of multiple topics as when jumping between topics evolving in parallel in a natural conversation so e.g.

* some question about topic A :A:
* some followup question :A:
* wait let's talk more about B now based on above :B:
* followup on that :B:
* so how does bla connect :A:B: <- now the full set of headings is relevant context

[johanvts]

@PalaceChan That's neat. But it requires either using org-mode to collect everything with the specified tag or implementing a similar functionality.

[PalaceChan]

hmmm i see - when i first started playing with gptel I wondered why even have the markdown option when org is so powerful that markdown mode support could be a limitation. but maybe i'm too org biased. a related thought (which might be easier to support in markdown too) based on the comment on properties would be ability to express external context that way e.g. via file glob patterns (or in a similar vein gptel-send from marked files in dired or ibuffer). That could enable some really awesome chat sessions with, e.g., most if not all the src code in a repo you're working on as context..

(the gptel-send with context from marked files might even be cooler if it's more of a "gptel-start-chat-from" that, like magit, takes you to a "commit message" style buffer where you type your question (and instead of a diff you maybe see summary list of files you're sending as context) and C-c C-c "commit" launches you into the chat session buffer)

[karthink]

when i first started playing with gptel I wondered why even have the markdown option when org is so powerful that markdown mode support could be a limitation.

Adding markdown support was zero cost and there are many Emacs users who don't use Org, so it made sense. We don't need parity between the features supported by gptel in markdown and org.

would be ability to express external context that way e.g. via file glob patterns (or in a similar vein gptel-send from marked files in dired or ibuffer).

I'm not planning to add this right now but gptel provides an interface to write your own interaction paradigms, see gptel-request and the wiki on this repo for details.

[karthink]

• so how does bla connect :A:B: <- now the full set of headings is relevant context

This level of granularity is beyond the scope of gptel. I think having a single tag/property on a headline to denote that it's a conversation is a good balance for now.

[armindarvish]

For what it's worth I implemented this in my own workflow without knowing anything about this discussion here, by using org-todo label for it and then using org-element API to parse the buffer and find the boundaries of the topic. So find the sontent between two headings with todo-item of gptel. This way I don't have to worry about how many subheadings I have. I think for a general purpose use, using tags can achieve the exact same thing. By the way, I also added an update-title since in this workflow I have conversation organized in topics (which sends a prompt asking chatGPT what the title of this specific conversation is). I am pretty happy with the results. Here is a screen shot:

Note that I add "User:" to my messages. That way I can also pass "System" messages transparently (as in I will remember what I did later if I look back at this file).

[karthink]

I haven't had time to work on this yet, but it remains on the docket for gptel.

For what it's worth I implemented this in my own workflow without knowing anything about this discussion here

This is pretty neat. FWIW you can automate the "User: " part by setting the entry in gptel-prompt-prefix-alist.

[karthink]

I've added support for limiting context to the current heading to gptel -- for Org buffers only, so far. (Work on Markdown is pending.)

To use it, run M-x gptel-set-topic under an Org heading. This will prompt you for a topic name, which can be anything. Further queries under this heading (even under sub-headings) will be limited to it.

(Alternatively, you can set the "GPTEL_TOPIC" Org property with org-set-property, C-c C-x p, which does the same thing.)

Support for saving and restoring state is also added for Org mode buffers. Saving a gptel buffer will save gptel metadata as Org properties. Opening a file and turning on gptel-mode will restore the state if available.

[karthink]

I'm planning some additional UX changes, including the ability to save/restore state outside of gptel-mode. This feature is basically in the package now, so I'm closing this FR.