karthink / gptel

A simple LLM client for Emacs
GNU General Public License v3.0
1.3k stars 129 forks source link

Optionally strip PROPERTIES when sending requests from Org mode buffers? #408

Open weavermarquez opened 3 days ago

weavermarquez commented 3 days ago

As indicated in the dry runs, the system message is included twice when querying from a gptel buffer turned org file.

Expected behaviour:

The gptel :PROPERTIES: values, especially GPTEL_SYSTEM, should not be included in the user prompt.

Replication Steps:

  1. Ensure gptel--system-message is set.
  2. Create a chatbot temporary buffer e.g. *Claude*
  3. Save it as an org file; this will create a PROPERTIES drawer.
  4. Run interactive gptel-menu, Dry Run > Inspect query (Lisp) [1] or (JSON) [2]

Examples

[1] Lisp

(:model "claude-3-5-sonnet-20240620" :system "     Preface your responses with a relevant hashtag at the beginning of each response.

     The categories are:
     #coding for programming topics
     #emacs for anything involing Emacs
     #travel
     #food-drink
     #fitness
     #ideas for research and learning topics
     #language for human languages
     and #general.
" :stream :json-false :max_tokens 4096 :messages
[(:role "user" :content ":PROPERTIES:
:GPTEL_MODEL: claude-3-5-sonnet-20240620
:GPTEL_BACKEND: Claude
:GPTEL_SYSTEM: Preface your responses with a relevant hashtag at the beginning of each response.\\n\\n     The categories are:\\n     #coding for programming topics\\n     #emacs for anything involing Emacs\\n     #travel\\n     #food-drink\\n     #fitness\\n     #ideas for research and learning topics\\n     #language for human languages\\n     and #general.\\n
:GPTEL_MAX_TOKENS: 4096
:GPTEL_BOUNDS: nil
:END:

*** Testing")]
:temperature 1.0)

[2] JSON

{
  "model": "claude-3-5-sonnet-20240620",
  "system": "     Preface your responses with a relevant hashtag at the beginning of each response.\n\n     The categories are:\n     #coding for programming topics\n     #emacs for anything involing Emacs\n     #travel\n     #food-drink\n     #fitness\n     #ideas for research and learning topics\n     #language for human languages\n     and #general.\n",
  "stream": false,
  "max_tokens": 4096,
  "messages": [
    {
      "role": "user",
      "content": ":PROPERTIES:\n:GPTEL_MODEL: claude-3-5-sonnet-20240620\n:GPTEL_BACKEND: Claude\n:GPTEL_SYSTEM: Preface your responses with a relevant hashtag at the beginning of each response.\\n\\n     The categories are:\\n     #coding for programming topics\\n     #emacs for anything involing Emacs\\n     #travel\\n     #food-drink\\n     #fitness\\n     #ideas for research and learning topics\\n     #language for human languages\\n     and #general.\\n\n:GPTEL_MAX_TOKENS: 4096\n:GPTEL_BOUNDS: nil\n:END:\n\n*** Testing"
    }
  ],
  "temperature": 1.0
}
karthink commented 2 days ago

The system message is not being sent twice, as you can see from the :system parameter of the request. gptel sends the buffer contents as-is, which includes the system message since it's been written to the buffer.

So the question is if all :PROPERTIES: blocks should be stripped when constructing the user prompt. See prior discussion in #141 (specifically this comment and my response) and one possible workaround based on #325.