:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed inference
LocalAI version: 2.19.3
Environment, CPU architecture, OS, and Version: Win 11, AMD Ryzen 5 4500 6-Core Processor, RTX 3090.
Describe the bug I am trying to use a custom embedding model. File is in gguf, the type of model is bert. My YAML file looks like :
The file is ItaLegalEmb.gguf and it's correctly placed in the model folder. When calling the model
The answer is
"could not load model: rpc error: code = Unknown desc = failed loading model"
The debug shows
Additional context