search

aricma / knowledge-graph

The aricma Knowledge Graph is an innovative data management solution designed to address the challenges of changing file storage methods, data search and retrieval, featuring a flexible storage structure, and intelligent indexing.
0 stars 0 forks source link

Build a document management system #2

Open aricma opened 7 months ago

aricma commented 7 months ago

To sort and search all my documents I want to build a system leveraging LLMs, vector db, and cloud storage technology to build a system that I can upload data to. The system will read the files and index it based on its content. These indexes can then be used to easily search the data/documents and store them in a very efficient manner.

aricma commented 7 months ago

ℹ️ This is an interesting talk about the GitHub search engine works: https://www.youtube.com/watch?v=CqZA_KmygKw.

aricma commented 7 months ago

Here is a link from the stack overflow newsletter about text embeddings. A concept that we used to build the first version of the knowledge graph. It can always help to understand more.

https://stackoverflow.blog/2023/11/09/an-intuitive-introduction-to-text-embeddings/?utm_campaign=the-overflow-newsletter&utm_medium=email&utm_source=iterable

aricma commented 7 months ago

Here is a link to a blog age by byte byte go about how to implement a document indexing system with vector databases etc. https://blog.bytebytego.com/p/unlock-highly-relevant-search-with?utm_source=post-email-title&publication_id=817132&post_id=139285061&utm_campaign=email-post-title&isFreemail=true&r=276we2&utm_medium=email

aricma commented 6 months ago

Here is a link about enabling/using full homomorphic encryption on aws: https://aws.amazon.com/blogs/machine-learning/enable-fully-homomorphic-encryption-with-amazon-sagemaker-endpoints-for-secure-real-time-inferencing/