search

irthomasthomas / llm-jina

A jina.ai plugin for simonw's llm cli featuring search, fact checking, embedding, reranking and more
3 stars 0 forks source link

Propose changes to llm project to integrate different ML models and tools. #1

Open irthomasthomas opened 2 weeks ago

irthomasthomas commented 2 weeks ago

llm-jina project is working well, but responses are raw, and not currently logged. Before I go on with this plugin I am thinking of making changes to the main llm project. I want to support other models like, rerankers and classifiers.

I feel a database per model type would grow out of hand quickly. I strongly prefer to treat any request to an ml model as a prompt/response in the responses table.

The main change would be to add a model_type column to the responses table. This could contain: llm, classifier, reranker, etc. The first issue I guess is that conversation_id doesn't really make sense. I guess another option would be to have a single additional db for none LLM models. I should look at the embedding db and see if I can reuse that better that the logs db.

irthomasthomas commented 2 weeks ago

Or a new ml_responses table in the main logs db