search

knmlprz / ChatKNML

ChatKNML is a Python-based chatbot powered by Large Language Models (LLMs) with an integrated Discord interface designed to respond to queries related to the Machine Learning Science Club and Rzeszów university of technology.
MIT License
4 stars 1 forks source link

Api embeding as service #127

Closed Kleczyk closed 5 months ago

Kleczyk commented 7 months ago

embeding-api is working more info in README.md Closes #122

Kleczyk commented 6 months ago

To explain why I removed most of the files. In lama.cpp all llm models have a "built-in embedding model" I knew this but I thought that if we use a newer one then llm will work better and this was also said at the meetings but nothing could be further from the truth since a particular model benefited from a specific embedding model during training will work better with the one it was trained with. Using the lama.cpp server I have access directly to the embedding model itself as a separate EP. For this reason, I discarded the idea of creating a separate service for embedding.