Updating `DoomEmacs`/`gptel` and chaing lightly configuration => only garbage generated now...

liar666 commented 1 month ago

Hi,

Context:

I'm experimenting with both Ellma and gptel for the moment, as I find nice stuff in both.
Until recently, both gptel ans Ellama where working correctly
Sometimes I need my model (for the moment orionstar/orion14b-q4:latest) to generate non-English texts, and just asking to talk another language in my prompt does not work, so I'd like to customize the pre-prompts ("system" prompts?directives?)
I recently upgraded my whole DoomEmacs

I also added this funciton in my config, that I call when I switch language globally (for grammar and for various LLMs):

(defun set-language-gptel (lang)
"Set language to french/english/portuguese for gptel LLMs"
(interactive)
(cond ((string= lang "british")
     (setq gptel-directives
           '(
            (default . "You are a large language model living in Emacs and a helpful assistant.  Respond concisely.")
            (programming . "You are a large language model and a careful programmer. Provide code and only code as output without any additional text, prompt or note.")
            (writing . "You are a large language model and a writing assistant. Respond concisely.")
            (chat . "You are a large language model and a conversation partner. Respond concisely.")
           )
     )
    )
    ((string= lang "francais")
     (setq gptel-directives
           '(
            (default . "Tu es un LLM vivant dans Emacs et un assistant utile. Réponds de manière concise.")
            (programming . "Tu es un LLM et un développeur attentionné. Fournit du code et seulement du code en sortie, sans texte additionnel, prompt ou note.")
            (writing . "Tu es un LLM et un assistant d'écriture. Réponds de manière concise.")
            (chat . "Tu es un LLM et un partenaire de conversation. Réponds de manière concise.")
           )
     )
    )
    ((string= lang "pt_BR")
     (setq gptel-directives
            '(
             (default . "Você é um LLM vivo em Emacs e um ajudante útil. Responda de forma concisa.")
             (programming . "Você é um LLM e um desenvolvedor atencioso. Forneça código e apenas saída de código, sem texto, prompts ou notas adicionais.")
             (writing . "Você é um LLM e assistente de redação. Responda de forma concisa.")
             (chat . "Você é um LLM e um parceiro de conversa. Responda de forma concisa.")
            )
     )
   )
)
)

Now, I get only garbage generated by gptel-send, whereas ellama-ask-selection still works perfectly...

Example:

generate the code of a tikz picture with 3 rectangles in blue 5 diamonds in red on their right and below a square of 3cm height with some lorem ipsum text in it

. The code that is called "lorem ipsum text on one side, so I need to produce?

=s with the code snippet of code `tikz-like this code.
Here you can be a blue. Here.

code. To.

What could be the problem?

liar666 commented 1 month ago

Before you ask, the rest of my config is pretty basic and has not changed:

  :config
  ;; Start server with ./llava-v1.5-7b-q4.llamafile --server --port 8000
  (gptel-make-openai "llama-cpp"          ;Any name
    :stream t                             ;Stream responses
    :protocol "http"
    :host "localhost:8000"                ;Llama.cpp server location
    :models '("test")
  )
  ;; OPTIONAL configuration
  (setq-default
        ;; HOME
        ;; gptel-model "gemma:2b"                        ; Pick your default model
        ;; WORK
        gptel-model "orionstar/orion14b-q4:latest"                        ; Pick your default model
        gptel-backend (gptel-make-ollama "gptel-localmodel" ; Any name of your choosing
                           :host "localhost:11434"          ; Where it's running
                           :stream t                        ; Stream responses
                           :models                          ; List of installed models
                           '(
                             ;; Code Specific Models
                             "codellama:latest"
                             "deepseek-coder:latest"
                             "stable-code:latest"
                             "starcoder:latest"
                             ;; Tiny Models
                             "dolphin-phi:latest"
                             "gemma:2b"
                             "orca-mini:latest"
                             "phi:latest"
                             "qwen:0"
                             "qwen:1"
                             "qwen:latest"
                             "stablelm-zephyr:latest"
                             "stablelm2:latest"
                             "tinydolphin:latest"
                             "tinyllama:latest"
                             ;; Medium Models
                             "gemma:7b"
                             "llama2-uncensored:latest"
                             "mistral:latest"
                             "vicuna:latest"
                             "zephyr:latest"
                             ;; Big Models
                             "mixtral:latest"
                             "llava:latest"
                             "dolphin-llama3:latest"
                             "orionstar/orion14b-q4:latest"
                             )
                      )
  )
  :bind
  (:map global-map
    ("C-S-g" . #'gptel-send)
  )
)

liar666 commented 1 month ago

FYI, here's what I get in Ellama. Not perfect but way better:

** User:
generate the code of a tikz picture with 3 rectangles in blue 5 diamonds in red on their right and below a square of 3cm height with some lorem ipsum text in it

** Ellama:
Here' an example of a TikZ picture with three blue rectangles, five red diamonds, and a square below it filled with 'lorem ipsum' text: 
#+END_SRC
\begin{tikzpicture} 
    \foreach \i in {1,2,3}
        \draw[blue, rounded corners, draw=black] (\i*4-4,0) rectangle (+(-2,3));

    \foreach \i in {1,...,5}
        \draw[red]    (3.5+\i,1) node[octagon, minimum size=2cm] {};

    \draw[blue, square, very thick] (-4,-4) -- (-4,0) -- (0,0) -- (0,-4);

    \node at (-6.25,-2.5) {\textcolor{black}{\begin{paragraph}Lorem ipsum dolor sit amet,  \textbf{consectetur adipiscing elit}, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.\end{paragraph}}}; 
\end{tikzpicture}
#+END_SRC

karthink commented 1 month ago

What could be the problem?

If you are using gptel in Org mode, could you try text-mode or markdown-mode? I suspect it's an issue with the Org conversion that gptel does.

karthink commented 1 month ago

Also please try the following:

(setq gptel-log-level 'info)
Use gptel with Ollama
Check the *gptel-log* buffer. You can paste its contents here.

liar666 commented 1 month ago

What could be the problem?

If you are using gptel in Org mode, could you try text-mode or markdown-mode? I suspect it's an issue with the Org conversion that gptel does.

Result in fundamental-mode is exactly the same :(

Can you explain in formal but simple terms for a newbie what indoor bouldering is and how to proceed once you are in front of a wall?
Can you explain what the wall looks like and the principle with colored holds

 on an indoor climbing gym? What should be done in it?
Thank you. Please also provide some tips that beginner climbers are beginners? How do I want to use in your first time or what is involved in terms of how to practice at a beginner, what would you??
-. if when is and? with on an what
 the for??

? it can of it; of in. to and? you it
 ins: -?
 ?? and the is are to and? or is please
. , beginners? a beginner in in. that in be
? in .

?? what as your an. you in a? for use in in a have, what of what? b,. how it. and? when you b-. can on a an to are how. What if to with in?
 are the to help a what is an? to go we can understand? I will be available? When you could you'x? You is a you is it has been explained for example you needs like a get you give a explain in order you have some are able to use on your trywareveryncectional, so the basic The next when i? And what is looking a and make an other be more understanding is one-excessive b understand they say that I could be sure it can look them but you to climb if you know how is given in this means? is there you have fun some better of your said. Tokat
Example forks,
in the basics and the following to go on a are interested?s?

liar666 commented 1 month ago

Also please try the following:

1. `(setq gptel-log-level 'info)`

2. Use gptel with Ollama

3. Check the `*gptel-log*` buffer.  You can paste its contents here.

Result is again BS:

Can you explain in formal but simple terms for a newbie what indoor bouldering is and how to proceed once you are in front of a wall?

Indoor bouldering is like?

Indoor Bouldering or rock climbing involves and what gear is needed and the process?
-the best way to understand it would be a beginner? I want to do the basic terms, please describe what the basics.

Begining of *gptel-log* ssems to show coorrect instruction are sent, but B is returned:

{
  "gptel": "request body",
  "timestamp": "2024-05-21 10:07:58"
}
{
  "model": "orionstar/orion14b-q4:latest",
  "messages": [
    {
      "role": "system",
      "content": "You are a large language model living in Emacs and a helpful assistant.  Respond concisely."
    },
    {
      "role": "user",
      "content": "Can you explain in formal but simple terms for a newbie what indoor bouldering is and how to proceed once you are in front of a wall?"
    }
  ],
  "stream": true,
  "options": {
    "temperature": 1.0
  }
}
{
  "gptel": "response body",
  "timestamp": "2024-05-21 10:08:11"
}
{
  "model": "orionstar/orion14b-q4:latest",
  "created_at": "2024-05-21T08:07:58.550080196Z",
  "message": {
    "role": "assistant",
    "content": "\n"
  },
  "done": false
}

Any idea what I could do wrong?

liar666 commented 1 month ago

OK I'm so sorry... Apparently it's the orion model that outputs bullshit... I just tried it on its own:

$ ollama run orionstar/orion14b-q4:latest
>>> Can you explain in formal but simple terms for a newbie what indoor bouldering is and how to proceed once you are in front of a wall?
... 

Indoor bouldering means?

Sure! Indoor bouldering is like? What do I need to know more about this term "lead climbing wall? 😊

I naïvely thought a larger model would be better... But finally using dolphin-llama3:latest give way better results!

liar666 commented 1 month ago

OMG... And I'm really a dumbass... I had already configured ellama with dolphin-llama3:latest, thus the difference in answer quality...

karthink / gptel

Updating `DoomEmacs`/`gptel` and chaing lightly configuration => only garbage generated now... #316