Loading a custom GGUF model

[topiga]

Is your feature request related to a problem? Please describe. Yes, this feature request is related to a problem (maybe specific to my use case). I can't load custom models from Alpaca.

Describe the solution you'd like An easy way to import a model, in a GGUF format. Maybe in the top left corner, a button to import a custom model.

Describe alternatives you've considered Using ollama, but nah, this app is fantastic (thanks btw)

[Jeffser]

Sounds good, I'm going to work on it for today's update

[Jeffser]

Hi, I kinda made it, it's still in testing but I will release it anyways, I'll polish it tomorrow

https://github.com/Jeffser/Alpaca/commit/c579e6ec999e5948584226cff6671f9748394a04

[tsilvs]

Can't load GGUFs from HuggingFace cloned git-lfs repos. Getting an error.

Here's the repo:

tree

``` ├── config.json ├── mistral-7b-instruct-v0.2.Q2_K.gguf ├── mistral-7b-instruct-v0.2.Q3_K_L.gguf ├── mistral-7b-instruct-v0.2.Q3_K_M.gguf ├── mistral-7b-instruct-v0.2.Q3_K_S.gguf ├── mistral-7b-instruct-v0.2.Q4_0.gguf ├── mistral-7b-instruct-v0.2.Q4_K_M.gguf ├── mistral-7b-instruct-v0.2.Q4_K_S.gguf ├── mistral-7b-instruct-v0.2.Q5_0.gguf ├── mistral-7b-instruct-v0.2.Q5_K_M.gguf ├── mistral-7b-instruct-v0.2.Q5_K_S.gguf ├── mistral-7b-instruct-v0.2.Q6_K.gguf ├── mistral-7b-instruct-v0.2.Q8_0.gguf └── README.md ```

Here's how it looks like in the file picker in Alpaca:

And here's what happens every time I try to load a .gguf file:

Error

$> flatpak run com.jeffser.Alpaca
Starting Alpaca's Ollama instance...
Started Alpaca's Ollama instance
[GIN] 2024/06/05 - 14:59:43 | 200 |      83.043µs |       127.0.0.1 | GET      "/"
[GIN] 2024/06/05 - 14:59:43 | 200 |     452.025µs |       127.0.0.1 | GET      "/api/tags"
[GIN] 2024/06/05 - 14:59:48 | 200 |     243.962µs |       127.0.0.1 | GET      "/api/tags"
/app/share/Alpaca/alpaca/window.py:376: Warning: g_value_get_int: assertion 'G_VALUE_HOLDS_INT (value)' failed
  self.create_model_name.set_text(name)
[GIN] 2024/06/05 - 15:00:07 | 200 |    1.766126ms |       127.0.0.1 | POST     "/api/create"
[GIN] 2024/06/05 - 15:00:07 | 200 |     173.282µs |       127.0.0.1 | GET      "/api/tags"
Stopped Alpaca's Ollama instance
Starting Alpaca's Ollama instance...
Started Alpaca's Ollama instance
**
Adwaita:ERROR:../src/adw-dialog-host.c:164:dialog_closing_cb: assertion failed: (g_ptr_array_find (self->dialogs, dialog, &index))
Bail out! Adwaita:ERROR:../src/adw-dialog-host.c:164:dialog_closing_cb: assertion failed: (g_ptr_array_find (self->dialogs, dialog, &index))

[Jeffser]

I'll take a look at that later today, thanks for reporting it