Closed halpz closed 1 year ago
this is a pytorch model, and is not compatible
Is there a handy way to convert Erebus from a pytorch model to a format supported by KoboldCPP? The readme says that GPT-NeoX models are supported, and Erebus is apparently a GPT-NeoX model.
It depends on which version of erebus. We do offer a gpt neoX 20b erebus here https://huggingface.co/KoboldAI/GPT-NeoX-20B-Erebus-GGML
However some of the smaller erebus models are OPT models instead of neoX which is not supported.
Thanks for the help! It's funny that you call the other Erebus models "smaller", the one I grabbed was split across 23 .bin files lmao
I downloaded a model from hugging face koboldai: and it doesn't seem to work:
System Info: AVX = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | WASMSIMD = 0 | BLAS = 1 | SSE3 = 0 | VSX = 0 | llama.cpp: loading model from /Users//Desktop/KoboldAI_OPT-2.7B-Erebus/pytorch_model.bin error loading model: unknown (magic, version) combination: 04034b50, 08080000; is this really a GGML file? llama_init_from_file: failed to load model llama_loadmodel: error: failed to load model '/Users//Desktop/KoboldAI_OPT-2.7B-Erebus/pytorchmodel.bin' Load Model OK: False Could not load model: /Users//Desktop/KoboldAI_OPT-2.7B-Erebus/pytorch_model.bin
is this model not compatible? if not where can I find a compatible version?