Model no longer supported - Launch error

[pomazanbohdan]

The interface starts, and after entering the first request, it crashes

PS C:\Users\pomazan> catai  serve
$ cd C:\Users\pomazan\AppData\Roaming\npm\node_modules\catai
$ npm start -- --production true --ui catai

> catai@0.3.8 start
> node src/index.js --production true --ui catai

llama.cpp: loading model from C:\Users\pomazan\catai\models\Alpaca-13B
llama_model_load_internal: format     = ggjt v1 (pre #1405)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 1024
llama_model_load_internal: n_embd     = 5120
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 40
llama_model_load_internal: n_layer    = 40
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: n_ff       = 13824
llama_model_load_internal: n_parts    = 1
llama_model_load_internal: model size = 13B
error loading model: this format is no longer supported (see https://github.com/ggerganov/llama.cpp/pull/1305)
llama_init_from_file: failed to load model
Listening on http://127.0.0.1:3000
new connection
llama.cpp: loading model from C:\Users\pomazan\catai\models\Alpaca-13B
llama_model_load_internal: format     = ggjt v1 (pre #1405)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 1024
llama_model_load_internal: n_embd     = 5120
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 40
llama_model_load_internal: n_layer    = 40
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: n_ff       = 13824
llama_model_load_internal: n_parts    = 1
llama_model_load_internal: model size = 13B
error loading model: this format is no longer supported (see https://github.com/ggerganov/llama.cpp/pull/1305)
llama_init_from_file: failed to load model
    at file:///C:/Users/pomazan/AppData/Roaming/npm/node_modules/catai/scripts/cli.js:69:27
    exit code: 1

[ido-pluto]

It seems that this format is no longer supported, that is a bit strange. I will look into that

[pomazanbohdan]

[ido-pluto]

Maybe this is with a newer quantization format. make sure you run the latest version of catai, a support to a new format will be added as soon as node-llama updates the llama.cpp version.

I will also look into this :)

[ido-pluto]

You can change the config to support alpaca with alpaca.cpp binding,

catai config --edit code

export const SELECTED_BINDING = 'alpaca-cpp';

BTW: The alpaca.cpp binding is outdated and slow, so it will not be included in the feature version.