Migrating to Verba: Retrieval Augmented Generation (RAG) chatbot powered by Weaviate

[sabszh]

Will update this post with evaluation as to whether it might make sense to migrate onto Verba, as well as where it could then be hosted.

• Link to Git rep: https://github.com/weaviate/Verba
• The UI: https://verba.weaviate.io/

[sabszh]

(IN PROGRESS...)

Hosting options

We need to host somewhere else than Streamlit cloud, as the app grows we need to be able to scale as we go and have a platform that can allow for the scale.

Pros, cons and time estimations for services

Snowflake

Pros

• Option for streamlit implementation / no need to spend time on frontend ]
• Cloud, hosting and database (data warehousing) with vector embeddings
• GDPR compliant
• Pay as you go

Cons

• Uses cloud through either Google, Amazon (buying into Sillicon valley... where does our data go hmmm)
• Can get pricey:

  "For any data company, Snowflake is already one of the largest, if not the largest, costs to manage. Every data team knows that Snowflake can become very expensive very fast." (Gilyadov, 2024)

• Might not be the best for our usecase:

  _"Even if you accept Snowflake's technical limitations as an application backend, you would still need to build and deploy a web service to serve the data. Snowflake can’t publish a query as a “REST API”." (Gilyadov, 2024)

Ploomber

Pros

• Streamlit compatibility
• Open source
• Branded as the heroku for AI applications

Cons

• Limited ecosystem
• Maturity