Kagi fastgpt using programmer system directive provides text/explanation even when told not to

[oscarvarto]

Example: if I provide the following instruction in a Scala buffer:

 // Write a scala function that computes the nth Fibonacci number using tail recursion.

I get the following output next:

Here is a Scala function that computes the nth Fibonacci number using tail recursion:

@tailrec
def fibonacci(n: Int, acc: (Int, Int)): Int = {
  if (n == 0) acc._1 
  else fibonacci(n-1, (acc._2, acc._1 + acc._2))
}

fibonacci(5, (0,1))

Output contains the text/explanation Here is a Scala function that computes the nth Fibonacci number using tail recursion: even when the system directive is You are a large language model and a careful programmer. Provide code and only code as output without any additional text, prompt or note.

Maybe there is something special about the way gptel is processing fastgpt responses, because this does not happen with gpt-3.5-turbo. However, I have to be very explicit and tell "do not wrap the output in a markdown code block" (when I am in a code buffer).

Some other small thing. Output is not formatted/indented properly according to the rest of the surrounding code.

[oscarvarto]

Note: I have checked that if I use a chat buffer, the output is code only, as expected.

[oscarvarto]

@karthink I'd like to use gptel with ChatGPT from OpenAI. I understand that I have to pay, but I am not sure if I have to pay for API calls only, or simply pay the $20 USD for the ChatGPT Plus Plan, or ... maybe both?

NOTE: I have realized I can pay for the API calls only. Thanks

[oscarvarto]

This works with Kagi's fastgpt:

  // !code Write a function that computes the nth Fibonacci number using tail recursion

Using the following (more explicit) directive:

You are a large language model and a careful Scala programmer. Provide code and only code as output without any additional text, prompt or note. Do not wrap output in a markdown code block.

Then I get a simple answer like

def fib(n: Int, current: Int = 0, next: Int = 1): Int = {
  if (n == 0) current 
  else fib(n-1, next, current + next)
}

NOTE: The output code is not indented properly, though.

[daedsidog]

Models are oftentimes dumb and don't know how to format resulting code. You should do all the formatting on your own, programmatically.

There is an active pull request of mine (https://github.com/karthink/gptel/pull/256) which adds a code formatter to the rewrite response.

[karthink]

Maybe there is something special about the way gptel is processing fastgpt responses, because this does not happen with gpt-3.5-turbo. However, I have to be very explicit and tell "do not wrap the output in a markdown code block" (when I am in a code buffer).

Some LLMs (like GPT-*) are tuned to pay more attention to system messages. Fastgpt does not even have the concept of a system message, gptel just includes it with the message. This could be the reason.

gptel is a thin shim for the LLM responses, it simply prints it as it is received. You can include whatever instructions ("don't wrap in markdown code blocks") you need in a system message and reuse it. The more explicit you are about your preferences the better the results tend to be.

Note: I have checked that if I use a chat buffer, the output is code only, as expected.

There is no difference between a chat and a code buffer as far as gptel's or the LLM's behavior is concerned. Have you checked that the code buffer is using the exact same system message as the chat buffer? Each buffer can have its own system message.

NOTE: The output code is not indented properly, though.

Try

(add-hook 'gptel-post-response-functions #'indent-region)

[karthink]

If the gptel-post-response-functions code works, please let me know.

[karthink]

Closing this since there is nothing to fix in gptel. Please reopen if the suggestion for indenting doesn't work.