unrecognized tensor type 4 on vicuna 13b uncensored model

[crypticpi]

Describe the bug I get this error trying to use the vicuna 13b uncensored model

llama.cpp: loading model from /Users/jvisker/catai/models/Vicuna-13B-Uncensored
error loading model: unrecognized tensor type 4

llama_init_from_file: failed to load model
Listening on http://127.0.0.1:3000
node:internal/process/promises:288
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[Error: Failed to initialize LLama context from file: /Users/jvisker/catai/models/Vicuna-13B-Uncensored]{
  code: 'GenericFailure'
}

Desktop (please complete the following information):

• OS: macOS m1
• Browser: Chrome
• CatAI version: 0.3.10
• Node.js version: 18.16.0

It works great on the 7B one

[ido-pluto]

This new model format will be supported in the next version of CatAI. In the meantime, you can install the old format:

catai install https://huggingface.co/eachadea/ggml-vicuna-13b-1.1/resolve/819b698c54b906ca7f3e1774cf748c2018932b07/ggml-old-vic13b-uncensored-q4_2.bin

Thanks for pointing this out, we will have a better solution to format installation in the feature

[crypticpi]

I get the same error on that model as well

[ido-pluto]

Seems that the link is broken, this one should work:

catai install https://huggingface.co/eachadea/ggml-vicuna-13b-1.1/resolve/819b698c54b906ca7f3e1774cf748c2018932b07/ggml-vic13b-uncensored-q4_0.bin

[ido-pluto]

The model links are now updated, so this issue should be fixed until llama.cpp can handle ggml v3.