[PGVector] Allow user to use their own database

[gozineb]

What

Specifically for vectors :

• Self Hosting vector databases
• Be compatible with every SQL database
• Allow user to choose which db she uses locally

How

need to change the way we interact with supabase

Dbs to use (opensource):

• milvus
• Chromadb

Create an intermediate layer ; Database (api keys, etc) Supabase Class that extends Database (match_vectors and other standalone functions)

[rodgermoore]

Elasticsearch would be nice too. For me this functionality (offline db and llms) is a total must. Privacy first. Great project though!

[hsm207]

Support for weaviate would be nice too 😀

[github-actions[bot]]

Thanks for your contributions, we'll be closing this issue as it has gone stale. Feel free to reopen if you'd like to continue the discussion.

[gwpl]

Self hosting is so critical feature! Maybe ChromaDB or any other solution woudl be allowed instead of relying on Supabase ?

[gwpl]

Btw. arised from discussion on LinkedIn : https://www.linkedin.com/feed/update/urn:li:activity:7098969909770014720?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A7098969909770014720%2C7099621509060509696%29&dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287099621509060509696%2Curn%3Ali%3Aactivity%3A7098969909770014720%29

[EvangelosSyrmos]

In order for the project to be fully open-source a self-hosted SQL database is required. Provide the option to include the necessary keys and credentials for a self-hosted database in the build step.

Although Supabase I built on top of open-source software, it is still a vendor lock-in option, thus limiting the potential for Quivr in the long run! Moreover, providing the option for a self-hosted database reduced the overall cost of running Quivr in a local network.

[giovannicocco]

With Milvus or Pinecone vector database will be great! Nice project btw.

[laurivaltteri]

Following this, for private psql!

[github-actions[bot]]

Thanks for your contributions, we'll be closing this issue as it has gone stale. Feel free to reopen if you'd like to continue the discussion.

[gwpl]

"Self hosting" topic is still very relevant !

Is there fully "self hosting" option? Or at least in terms of vector store? And to choose AI providers?

[gwpl]

Any progress on self hosted setup?

[StanGirard]

@gwpl checkout the new readme :)

[gwpl]

I see ollama.ai alternative mentioned! Nice! Thank you!

Still I am not sure if I understand that there is no mention of "vector" or "store" in current readme, as it's automatically self hosted deployed? What vectore store is used now? (is it supabase?) Maybe worth to mention in README.md what vector store is used and if it's run inside docker (compose ) ?

[Jean-Baptiste-Lasselle]

Hello all who messaged here, I want to add up, to make it as clear as possible for the project team :

Yes of course being able to fully operate completely offline from pure open source, is a must be feature

But much more than that : I am sure others did too, now If i took any minute to look at the project, that is because I am looking for open source project for AI, getting rid of any third party dependencies.

Open AI like all of any AI products are all completely forbidden in all the biggest corporations i work for and everyone jnows why in our professions, with on top the fact i know all know: the 0% percent knowledge managers we all have in corporations are even much harder than us, on the "no third party AI", you would even risk to be fired badly if you ever fail to comply to that hard rule. My point it is time to just acknowledge the reality, the world will continue incredible things, with or without AI, and AI not any revolution at all but based on classical not important mathematics which didnt make any relevant progress since the 1970s. The only thing is that there is a potential business value with them today, given the change of hardware between the 1970s and today.

Redhat sells linux and became world leaders in their industry, yet all is open source : for the sake of the project, please be sure you will find features like security related, which you will be able to sell for producton ready environments in huge corporations. Making the project opensource will only bring you more help from the entire world.

Now like all projects, I will say like I have said many times: If they don't do it fully open source, that's because there is nothing at all, if you don't get something up n running just by copy pasting and following Docs instructions, don't spend one more second reading any of it, IT is engineering, you will never ever get deep ideas from there, deep ideas come from research and pure artists, if you don't understand anything from those reseachers and artists, you won't understand anything at all more by listening to engineers. Engineering is the art of making things work, not of discovering or explaing ideas.

It's exactly like years ago, when i heard some people say "Oh i am so interested in algoritmic theories, that's why i do dev", I just laughed, and to make my point I stated how funny it is to find people saying they are so interested in algorithm theories for years, yet they are not even able to give the definition of the "développement limité à l'ordre N, d'une fonction, au voisinage d'un point".

Just ask any engineer in your team to calculate the "développement limité d'ordre trois, au voisinage du point zéro" of the most simple C1 function, and you will see.

We will see about this project, like thousands of other we keep an eye on, and I can't wait to see dosu's answer to this comment , and I send best regards to my pairs here, with a ps https://github.com/StanGirard/quivr/issues/1846

[github-actions[bot]]

Thanks for your contributions, we'll be closing this issue as it has gone stale. Feel free to reopen if you'd like to continue the discussion.

[gwpl]

Maybe it should be more active to not be all the time auto-closed by bot?

[mckbrchill]

Qdrant is a good option too, alongside with Weaviate. Also, these open source vector DBs are much faster and a bit more accurate than pgvector

[github-actions[bot]]

Thanks for your contributions, we'll be closing this issue as it has gone stale. Feel free to reopen if you'd like to continue the discussion.

[StanGirard]

We are currently working on that :)