search

karthink / gptel

A simple LLM client for Emacs
GNU General Public License v3.0
1.04k stars 113 forks source link

json.exception.type_error.316: invalid UTF-8 byte at index 23: 0x28 #164

Open fast-90 opened 6 months ago

fast-90 commented 6 months ago

I am getting an error when I specifically use deepseek-coder:6.7B-base with Ollama. Not sure if it is an issue with the deepseek-coder model or with gptel, but I thought I would try to ask here since I can get the model working with gptel under very specific situations.

The error specifically occurs when:

  1. I select a line with Python code (in my example: def sort_array(array):.
  2. Run gptel with deepseek-coder:6.7B-base (directives/parameters are set to default values).

The error does not occur if I:

  1. Prompt the same Python code in a gptel (*ChatGPT*) buffer (this produces the expected output).
  2. Prompt the same Python code directly to ollama in the terminal (ollama run deepseek-coder:6.7B-base 'def sort_array(array):')
  3. Try the same with a different model.

Error messages

Messages buffer

("--disable" "--location" "--silent" "--compressed" "-XPOST" "-y300" "-Y1" "-D-" "-w(205c14fbcbcafb4eb97f2c322b1590c3 . %{size_header})" "-d{\"model\":\"deepseek-coder:6.7b-base\",\"system\":\"You are a large language model living in Emacs and a helpful assistant. Respond concisely.\",\"prompt\":\"def sort_array():\",\"stream\":true}" "-HContent-Type: application/json" "http://localhost:11434/api/generate")
("--disable" "--location" "--silent" "--compressed" "-XPOST" "-y300" "-Y1" "-D-" "-w(487e8ed6e02318be4cfa6a331a340a46 . %{size_header})" "-d{\"model\":\"deepseek-coder:6.7b-base\",\"system\":\"\",\"prompt\":\"def sort_array():\",\"stream\":true,\"context\":[770,5166,29901,13,1678,822,903,2344,12035,1311,29892,1361,29901,851,1125,13,4706,1583,29889,3179,353,1361,13,4706,1583,29889,2798,29879,353,315,5336,29898,3179,29897,13,4706,1583,29889,13092,353,7431,29898,842,29898,3179,876,13,4706,1583,29889,3317,29918,2798,353,1583,29889,657,29918,3317,29918,2798,580,13,13,1678,822,679,29918,3317,29918,2798,29898,1311,29897,1599,938,29901,13,4706,736,1583,29889,2798,29879,29961,3317,29898,1311,29889,2798,29879,4638,13,13,1678,822,1423,29918,20818,29918,974,29918,29874,29918,14380,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,13092,1275,29871,29896,13,13,1678,822,1423,29918,17823,29918,974,29918,29874,29918,14380,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,3317,29918,2798,1275,29871,29946,13,13,1678,822,1423,29918,8159,29918,8697,29898,1311,29897,1599,6120,29901,13,4706,736,313,1311,29889,3317,29918,2798,1275,29871,29941,29897,322,313,1311,29889,13092,1275,29871,29906,29897,13,13,1678,822,1423,29918,17536,29918,974,29918,29874,29918,14380,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,3317,29918,2798,1275,29871,29941,13,13,1678,822,1423,29918,10184,29918,29886,7121,29898,1311,29897,1599,6120,29901,13,4706,11000,353,518,29894,363,325,29892,274,297,1583,29889,2798,29879,29889,7076,580,565,274,1275,29871,29906,29962,13,4706,736,7431,29898,29886,7121,29897,1275,29871,29906,322,313,1311,29889,13092,448,29871,29896,29897,1275,29871,29906,13,13,1678,822,1423,29918,650,29918,18784,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,3317,29918,2798,1275,29871,29906,13,13,1678,822,679,29918,3179,29898,1311,1125,13,4706,565,1583,29889,3198,29918,20818,29918,974,29918,29874,29918,14380,7295,13,9651,736,5166,1542,16854,29889,3738,12064,29918,9800,29918,29909,29918,29968,22255,13,4706,25342,1583,29889,3198,29918,17823,29918,974,29918,29874,29918,14380,7295,13,9651,736,5166,1542,16854,29889,5800,4574,29918,9800,29918,29909,29918,29968,22255,13,4706,25342,1583,29889,3198,29918,8159,29918,8697,7295,13,9651,736,5166,1542,16854,29889,29943,3299,29918,8187,17171,13,4706,25342,1583,29889,3198,29918,17536,29918,974,29918,29874,29918,14380,7295,13,9651,736,5166,1542,16854,29889,4690,21661,29918,9800,29918,29909,29918,29968,22255,13,4706,25342,1583,29889,3198,29918,10184,29918,29886,7121,7295,13,9651,736,5166,1542,16854,29889,16240,29949,29918,7228,8193,29903,13,4706,25342,1583,29889,3198,29918,650,29918,18784,7295,13,9651,736,5166,1542,16854,29889,12413,29918,7228,8193,13,4706,1683,29901,13,9651,12020,8960,877,14126,12018,1361,1495,13,13,1990,5166,29901,13,1678,822,903,2344,12035,1311,29892,1361,29901,851,1125,13,4706,1583,29889,3179,353,1361,13,4706,1583,29889,2798,29879,353,315,5336,29898,3179,29897,13,4706,1583,29889,13092,353,7431,29898,842,29898,3179,876,13,4706,1583,29889,3317,29918,2798,353,1583,29889,657,29918,3317,29918,2798,580,13,13,1678,822,679,29918,3317,29918,2798,29898,1311,29897,1599,938,29901,13,4706,736,1583,29889,2798,29879,29961,3317,29898,1311,29889,2798,29879,4638,13,13,1678,822,1423,29918,20818,29918,974,29918,29874,29918,14380,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,13092,1275,29871,29896,13,13,1678,822,1423,29918,17823,29918,974,29918,29874,29918,14380,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,3317,29918,2798,1275,29871,29946,13,13,1678,822,1423,29918,8159,29918,8697,29898,1311,29897,1599,6120,29901,13,4706,736,313,1311,29889,3317,29918,2798,1275,29871,29941,29897,322,313,1311,29889,13092,1275,29871,29906,29897,13,13,1678,822,1423,29918,17536,29918,974,29918,29874,29918,14380,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,3317,29918,2798,1275,29871,29941,13,13,1678,822,1423,29918,10184,29918,29886,7121,29898,1311,29897,1599,6120,29901,13,4706,736,313,1311,29889,13092,6736,29871,29906,29897,322,313,1311,29889,657,29918,3317,29918,2798,580,2804,29871,29946,29897,13,13,1678,822,1423,29918,650,29918,18784,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,3317,29918,2798,1275,29871,29906,13,13,1678,822,1423,29918,9812,29918,7543,29898,1311,29897,1599,6120,29901,13,4706,736,5852,13,1990,5166,29901,13,1678,822,903,2344,12035,1311,29892,1361,29901,851,1125,13,4706,1583,29889,3179,353,1361,13,4706,1583,29889,2798,29879,353,315,5336,29898,3179,29897,13,4706,1583,29889,13092,353,7431,29898,842,29898,3179,876,13,4706,1583,29889,3317,29918,2798,353,1583,29889,657,29918,3317,29918,2798,580,13,13,1678,822,679,29918,3317,29918,2798,29898,1311,29897,1599,938,29901,13,4706,736,1583,29889,2798,29879,29961,3317,29898,1311,29889,2798,29879,4638,13,13,1678,822,1423,29918,20818,29918,974,29918,29874,29918,14380,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,13092,1275,29871,29896,13,13,1678,822,1423,29918,17823,29918,974,29918,29874,29918,14380,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,3317,29918,2798,1275,29871,29946,13,13,1678,822,1423,29918,8159,29918,8697,29898,1311,29897,1599,6120,29901,13,4706,736,313,1311,29889,3317,29918,2798,1275,29871,29941,29897,322,313,1311,29889,13092,1275,29871,29906,29897,13,13,1678,822,1423,29918,17536,29918,974,29918,29874,29918,14380,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,3317,29918,2798,1275,29871,29941,13,13,1678,822,1423,29918,10184,29918,29886,7121,29898,1311,29897,1599,6120,29901,13,4706,736,313,1311,29889,13092,6736,29871,29906,29897,322,313,1311,29889,657,29918,3317,29918,2798,580,2804,29871,29946,29897,13,13,1678,822,1423,29918,650,29918,18784,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,3317,29918,2798,1275,29871,29906,13,13,1678,822,1423,29918,9812,29918,7543,29898,1311,29897,1599,6120,29901,13,4706,736,5852,1990,5166,29901,13,1678,822,903,2344,12035,1311,29892,1361,29901,851,1125,13,4706,1583,29889,3179,353,1361,13,4706,1583,29889,2798,29879,353,315,5336,29898,3179,29897,13,4706,1583,29889,13092,353,7431,29898,842,29898,3179,876,13,4706,1583,29889,3317,29918,2798,353,1583,29889,657,29918,3317,29918,2798,580,13,13,1678,822,679,29918,3317,29918,2798,29898,1311,29897,1599,938,29901,13,4706,736,1583,29889,2798,29879,29961,3317,29898,1311,29889,2798,29879,4638,13,13,1678,822,1423,29918,20818,29918,974,29918,29874,29918,14380,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,13092,1275,29871,29896,13,13,1678,822,1423,29918,17823,29918,974,29918,29874,29918,14380,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,3317,29918,2798,1275,29871,29946,13,13,1678,822,1423,29918,8159,29918,8697,29898,1311,29897,1599,6120,29901,13,4706,736,313,1311,29889,3317,29918,2798,1275,29871,29941,29897,322,313,1311,29889,13092,1275,29871,29906,29897,13,13,1678,822,1423,29918,17536,29918,974,29918,29874,29918,14380,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,3317,29918,2798,1275,29871,29941,13,13,1678,822,1423,29918,10184,29918,29886,7121,29898,1311,29897,1599,6120,29901,13,4706,736,313,1311,29889,13092,6736,29871,29906,29897,322,313,1311,29889,657,29918,3317,29918,2798,580,2804,29871,29946,29897,13,13,1678,822,1423,29918,650,29918,18784,29898,1311,29897,1599,6120,29901,13,4706,736,1583,29889,3317,29918,2798,1275,29871,29906,13,13,1678,822,1423,29918,9812,29918,7543,29898,1311,29897,1599,6120,29901,13,4706,736,5852,13]}" "-HContent-Type: application/json" "http://localhost:11434/api/generate")
Ollama error: (HTTP/1.1 500 Internal Server Error) 500 Internal Server Error
[json.exception.type_error.316] invalid UTF-8 byte at index 23: 0x28

gptel error buffer

HTTP/1.1 500 Internal Server Error
Content-Type: application/json; charset=utf-8
Date: Sun, 31 Dec 2023 12:26:48 GMT
Content-Length: 107

{"error":"500 Internal Server Error\n[json.exception.type_error.316] invalid UTF-8 byte at index 23: 0x28"}(76e01982a8f9ea988ac2c85ba88815d5 . 143)

My OS / config

OS

macOS Sonoma 14.2.1, emacs-plus (29.1) installed via homebrew.

Hardware

Macbook M3 Pro (18 GB RAM)

Config

(use-package gptel
  :config
  (setq-default gptel-backend
                (gptel-make-ollama
                 "Ollama"
                 :host "localhost:11434"
                 :models '("mistral" "dolphin-mistral" "solar" "codellama" "codellama:7b-python" "deepseek-coder:6.7b-base")
                 :stream t)))
fast-90 commented 6 months ago

Update: I restarted Emacs and deepseek-coder:6.7b-base started to work as expected again. However, now I get the same issue as above with all other models (e.g. codellama). Same error messages.

karthink commented 5 months ago

Thanks for the report. I haven't been able to reproduce this with the models I've tried with Ollama (including deepseek-coder), but in the meantime I received additional reports from someone using gptel + Ollama + nous-hermes2.

Are you still getting this error reliably? If you are, I could test a couple of things on the Emacs side.