GPT4All: incomplete answers

[novoid]

Hi,

After reading your message on reddit I tested gptel with my GPT4All.

To enable this package to query GPT4All, you need to activate the GPT4All setting: Application → "Enable API server"

(use-package gptel
  :ensure t
  :config
  (setq-default gptel-model "mistral-7b-openorca.Q4_0.gguf" ;; see /home/$USER/.local/share/nomic.ai/GPT4All/
                gptel-host "localhost:4891"
                gptel-protocol "http"
                gptel-api-key "--"
                gptel-stream nil)
)

Unfortunately, the answers do seem to be incomplete:

My testing question was "What is the airspeed velocity of an unladen swallow?"

Marking that as a region and invoking gptel-send returned first "The airspeed velocity of an African Swallow (also known as Barn Swallow" and on the second run "The airspeed velocity of an unaden swallow is about 24 miles per".

With a dedicated gptel buffer, I got: "The question appears to be asking for the flying speed of a swift, often called".

In GPT4All GUI, I get: "An African Swallow can reach speeds up to 99 miles per hour (160 km/h). However, this speed may vary depending on factors such as altitude and weather conditions."

Maybe you do have an idea what I do wrong or where the issue is originated.

[karthink]

You need to increase the max_tokens parameter, gpt4all sets it very low (50). You can do this by setq-default gptel-max-tokens to (say) 400 in your config, or live from the transient menu you get by calling gptel-send with a prefix argument (C-u C-c RET)

The exact names might be different, please excuse the inaccuracy. I'm away from my computer right now.

On Sat, Oct 28, 2023, 12:03 PM Karl Voit @.***> wrote:

Hi,

After reading your message on reddit https://www.reddit.com/r/emacs/comments/17dvpej/gpt4all_mode_in_emacs/ I tested gptel with my GPT4All.

To enable this package to query GPT4All, you need to activate the GPT4All setting: Application → "Enable API server"

(use-package gptel :ensure t :config (setq-default gptel-model "mistral-7b-openorca.Q4_0.gguf" ;; see /home/vk/hosts/sting/config/dot-local/share/nomic.ai/GPT4All/ gptel-host "localhost:4891" gptel-protocol "http" gptel-api-key "--" gptel-stream nil) )

Unfortunately, the answers do seem to be incomplete:

My testing question was "What is the airspeed velocity of an unladen swallow?"

Marking that as a region and invoking gptel-send returned first "The airspeed velocity of an African Swallow (also known as Barn Swallow" and on the second run "The airspeed velocity of an unaden swallow is about 24 miles per".

With a dedicated gptel buffer, I got: "The question appears to be asking for the flying speed of a swift, often called".

In GPT4All GUI, I get: "An African Swallow can reach speeds up to 99 miles per hour (160 km/h). However, this speed may vary depending on factors such as altitude and weather conditions."

Maybe you do have an idea what I do wrong or where the issue is originated.

— Reply to this email directly, view it on GitHub https://github.com/karthink/gptel/issues/115, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACBVOLHK6OQ7KFDB6T7JY3DYBVJG3AVCNFSM6AAAAAA6UIUPYKVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE3DMNZQGY4TQMQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[karthink]

@novoid Just a heads-up. I completed the local LLM support feature for gptel, and the above config code (which was a hack) will no longer work.

Here's what you need now to register a gpt4all backend:

(gptel-make-gpt4all
 "GPT4All"                              ;Name of your choosing
 :protocol "http"
 :host "localhost:4891"                 ;Where it's running
 :models '("mistral-7b-openorca.Q4_0.gguf")) ;Available models

If you want this to be the default, instead of ChatGPT, you can set it as the default value of gptel-backend:

;; Make gpt4all the default
(setq-default
 gptel-backend
 (gptel-make-gpt4all
  "GPT4All"                             ;Any name of your choosing
  :protocol "http"                       
  :host "localhost:4891"                       ;Where it's running
  :models '("mistral-7b-openorca.Q4_0.gguf"))) ;Available models

;; Don't limit token count:
(setq-default gptel-max-tokens 800)

You may want to peruse the README to see the new features, in particular the multi-LLM demo near the top.

[novoid]

This is causing:

Error (use-package): gptel/:config: Symbol’s function definition is void: gptel-make-gpt4all

with 0.3.5 from elpa. I guess you're referring to a future version.

However, the original issue did vanish after increasing "gptel-max-tokens" so I'll close the issue for now.

Thank you very much!

[karthink]

Indeed, I was referring to the latest update. You'll need to update your configuration when you update gptel - there has been a pretty big change to the design so I can officially support GPT4All, Ollama etc.

On Sun, Oct 29, 2023, 5:48 AM Karl Voit @.***> wrote:

This is causing:

Error (use-package): gptel/:config: Symbol’s function definition is void: gptel-make-gpt4all

with 0.3.5 from elpa. I guess you're referring to a future version.

However, the original issue did vanish after increasing "gptel-max-tokens" so I'll close the issue for now.

Thank you very much!

— Reply to this email directly, view it on GitHub https://github.com/karthink/gptel/issues/115#issuecomment-1784098395, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACBVOLAIRUGICMNIZHIDTQDYBY7CFAVCNFSM6AAAAAA6UIUPYKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBUGA4TQMZZGU . You are receiving this because you commented.Message ID: @.***>