search

karthink / gptel

A simple LLM client for Emacs
GNU General Public License v3.0
1.03k stars 111 forks source link

Different results between gptel+ChatGPT and chat.openai.com #300

Closed dmitrym0 closed 2 months ago

dmitrym0 commented 2 months ago

Hey folks,

I'm attempting to use gptel to summarize my daily work.

Here's the partial prompt that I used:

This is a record of the day, in org-mode format. It starts with the * TODO or * DONE (this is a heading for the task). TODO Signifies that the task hasn't been attempted, while DONE identifies that some work was done.

The heading usually includes a project, a JIRA link and the description of the goal of the task.

JIRA links comes in the form of XXX-#### where #### is a numerical ID.

Each heading may have a note below it indicating what was done, difficulties encountered etc.

Go through the list:

  • summarize the projects that were worked on, including a jira ticket if it exists
  • if there are notes for a task, summarize these too.
  • separately, list the tasks that were not attempted.

If I combine this prompt with my daily org log and paste it into chat.openai.com I get what I want in the format described.

If I set the above as a prompt via gptel (gptel-system-prompt), and then use gptel-send while having my daily log highlighted, I do not get the expected output -- though the contents are essentially the same.

I see that GPT-3.5 is selected in gptel and chat.openai. What am I missing?

karthink commented 2 months ago

This description seems fine. You could check if what's being sent is exactly what you intend, in the following way:

  1. Run (setq gptel-expert-commands t)
  2. Set your system prompt, select your daily org log and bring up gptel's transient menu (C-u M-x gptel-send)
  3. Invoke one of the dry run options with I or J (Lisp or JSON) to examine the payload.
dmitrym0 commented 2 months ago

@karthink I went looking for dry run options 'cause they were not appearing in my transient menu. Turns out I was running version 0.3.5 from ~9 months ago. Updated and everything looks fine.

Thank you!