feat: Support for multilingual embedding models and simultaneous hosting of LLM & embedding models

janhq / cortex

Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM, ONNX). Powers 👋 Jan

https://cortex.so

Apache License 2.0

1.8k stars 98 forks source link

feat: Support for multilingual embedding models and simultaneous hosting of LLM & embedding models #562

Open eckartal opened 2 months ago

eckartal commented 2 months ago

Problem Jan lacks support for multilingual embedding models and cannot host both LLM and embedding models simultaneously.

Success Criteria Jan should be enhanced to support a broader range of multilingual embedding models. Plus, it should be capable of hosting both LLM and embedding models concurrently within a single system.

Additional context Add any other context or screenshots about the feature request here.

kalle07 commented 2 months ago

yes if iam right maybe that is a german type of such model

https://huggingface.co/aari1995/German_Semantic_STS_V2