search

ravenscroftj / turbopilot

Turbopilot is an open source large-language-model based code completion engine that runs locally on CPU
BSD 3-Clause "New" or "Revised" License
3.83k stars 127 forks source link

Failed to load model wizardcoder (Illegal Instruction) #70

Open Gnomesenpai opened 1 year ago

Gnomesenpai commented 1 year ago

Hello,

I am unable to launch this with the wizard coder model. Dell R620 server with Dual E5-2637v2, 128GB RAM. Using the v0.2.0 AVX build.

`root@lab:~/copilot# ./turbopilot --debug --model-file ./models/WizardCoder-15B-1.0.ggmlv3.q4_0.bin [2023-09-07 22:14:24.316] [debug] debug logging enabled [2023-09-07 22:14:24.316] [info] Initializing GPT-J type model for 'codegen' model [2023-09-07 22:14:24.316] [info] Attempt to load model from codegen [2023-09-07 22:14:24.316] [info] load_model: loading model from './models/WizardCoder-15B-1.0.ggmlv3.q4_0.bin' - please wait ...

[2023-09-07 22:14:24.316] [info] load_model: n_vocab = 49153

[2023-09-07 22:14:24.316] [info] load_model: n_ctx = 8192

[2023-09-07 22:14:24.316] [info] load_model: n_embd = 6144

[2023-09-07 22:14:24.316] [info] load_model: n_head = 48

[2023-09-07 22:14:24.316] [info] load_model: n_layer = 40

[2023-09-07 22:14:24.316] [info] load_model: n_rot = 2002

[2023-09-07 22:14:24.316] [info] load_model: ftype = 49153

[2023-09-07 22:14:24.316] [info] load_model: qntvr = 49

[2023-09-07 22:14:24.316] [error] load_model: invalid model file './models/WizardCoder-15B-1.0.ggmlv3.q4_0.bin' (bad vocab size 13 != 49153)

[2023-09-07 22:14:24.316] [error] Failed to load model`

root@lab:~/copilot# ./turbopilot --debug --model-type wizardcoder --model-file ./models/WizardCoder-15B-1.0.ggmlv3.q4_0.bin [2023-09-07 22:16:02.789] [debug] debug logging enabled [2023-09-07 22:16:02.789] [info] Initializing Starcoder/Wizardcoder type model for 'wizardcoder' model type [2023-09-07 22:16:02.789] [info] Attempt to load model from wizardcoder load_model: loading model from './models/WizardCoder-15B-1.0.ggmlv3.q4_0.bin' load_model: n_vocab = 49153 load_model: n_ctx = 8192 load_model: n_embd = 6144 load_model: n_head = 48 load_model: n_layer = 40 load_model: ftype = 2002 load_model: qntvr = 2 load_model: ggml ctx size = 25608.48 MB Illegal instruction