dsebastien
commented
2 weeks ago For V1, we're going to give Replicate.com a try. This should enable us to move faster.
Open dsebastien opened 2 weeks ago
dsebastien
commented
2 weeks ago For V1, we're going to give Replicate.com a try. This should enable us to move faster.
Goal: be able to leverage LLMs for AI features (text analysis, sentiment analysis, summarization, automatic tagging/categorization, etc).
Two main ways I think about:
Replicate.com is interesting, because we can pay per use, and don't need beefy hardware for Knowii itself. The tradeoff is that we will have to send our data over to them (vs GDPR, privacy, etc), and will be limited to the models they have. But in the vast majority of use cases I have in mind now, no private data will be involved, so it's not a show-stopper. It could be a solid first step towards integrating AI features in Knowii.
Ollama is great, but requires solid machines to install and use the models. The nice thing is that it offers a unified interface for many LLMs. Moreover, we will have to pay for those machines 24/7, whereas Replicate would enable us to pay per use instead.
References: