posts/intro-to-embeddings-vector-databases-and-rag/

[utterances-bot]

Cloud Report - Intro to Embeddings, Vector Databases, and RAG

An introduction to using large language models to search and summarize your data.

https://blog.caseyharalson.com/posts/intro-to-embeddings-vector-databases-and-rag/

[dythomas1]

Fascinating article. Thanks for sharing. How hard would it be to create LLM's with locally sourced vector database servers for businesses that store a lot of text like information? For example, Instead of going through all of the business's SOPs or documentation to get to the single username and password that you need to test a piece of new code about to be deployed, you would have a LLM retrieve it for you based on a prompt.

[CaseyHaralson]

@dythomas1 Thanks for your comment!

For your question, if we assume a certain documentation structure then this should already be pretty doable. For example, take a document for "Product A" like:

Title: Product A Testing Procedures

stuff

Test Environment Credentials
Username: testuser1
Password: *&^$@#abAB

stuff

If that document was stored in a vector database and you ask the database for the document closest to "Product A test environment credentials", then this document should be returned at or near the top of the list.

Now if we take that document and create a prompt to send to a LLM:

Using the following document, return the test environment username and password in json format like {"username":"username here", "password":"password here"}.

<document>
Title: Product A Testing Procedures

stuff

Test Environment Credentials
Username: testuser1
Password: *&^$@#abAB

stuff
</document>

I copy/pasted that query into ChatGPT and this is what it returned:

{
  "username": "testuser1",
  "password": "*&^$@#abAB"
}

If you haven't tried playing with a vector database yet and want a suggestion, try Weaviate. It has a lot of cool capabilities and helps get past the initial learning hump pretty quickly.

You might be interested in LangChain too.

[maqboolp]

Fantastic piece, Casey. The and tags you've introduced are essentially aids for organizing thoughts and enhancing clarity, correct? They don't inherently convey any special meaning to the ChatGPT language model, although they might offer some contextual hints. Thoroughly enjoyed the read; looking forward to more of your work.

[dythomas1]

Ok very cool Casey. Thank you. This will be something I will look into more. I'd like to understand current AI capabilities so that I can start building a vision for the future. Thanks again, Drew.

[CaseyHaralson]

@maqboolp Thank you!

The tags can actually be helpful to let ChatGPT know what it should be looking for when creating the system prompt. If you take a look at the OpenAI Prompt Engineering Guide, they delimit quite often with triple quotes or XML tags.

So they suggest doing something like:

Summarize the text delimited by triple quotes in about 50 words.

"""Some
long
text"""

Or:

You will be provided with a pair of articles (delimited with XML tags) about the same topic. First summarize the arguments of each article. Then indicate which of them makes a better argument and explain why.

<article> insert first article here </article>

<article> insert second article here </article>

I did use tags in this article to help with clarity though :)