I think the key part of making vector search work well is to consistently generate a snippet for storage and retrieval. For example, when you came across this HostedGPT post and you bookmark it, there is some process of committing this to memory, maybe we callMemorize.new.commit_link(url). In analyzing the link we construct a string such as:
Reddit post on Monday, May 13th on the rails subreddit announcing a new version of HostedGPT running GPT-4o. The post explains how to set it up and why it's better. It's about OpenAI, ChatGPT, Github, open source, quota, and ELO ranking.
This would be the string we turn into a vector. The key idea is that this string is generic by following a universal heuristic. The heuristic I just used is:
[source] post on [date with day of week] on the [outlet pretty_name] [post_summary]. It's about [post_keywords_not_mentioned_in_summary].
Note: The two hard parts about this encoding are the post_summary and the post_keywords. When we are committing something to memory, we will need to come up with special prompt that generates these. (I'll include an example below)
Then, after this has been committed to memory, the user may ask for retrieval in a variety of ways. For example, they might ask the AI, "What was that ChatGPT clone built on Rails that I found on Reddit?" and they also might ask, "I saw a project last week on one of the platforms, I forget which one, it was an open source chat bot. Where did I read about that?" We don't use those exact strings for vector matching, instead we use an LLM to convert this question into our "snippet" format. The reason we're mapping things to a universal snippet is to help account for such differences in input text when they're essentially asking for the same question. To continue these examples:
Input: What was that ChatGPT clone built on Rails that I found on Reddit?
Snippetized: Reddit post on the subreddit about a ChatGPT clone built in Rails
Input: I saw a project last week on one of the platforms, I forget which one, it was an open source chat bot. Where did I read about that?
Snippetized: Post around Sunday, May 12 to Saturday, May 18 about an open source chat bot
When we are committing things, this is an example of a prompt which might generate the keywords and summary that we want:
We are going to ask a person to read this post and then afterwards they are going to tell us a list of topics mentioned in the post and then a summary of the post. For example, when they read this post from the rails subreddit on Reddit:
[[[
New version of HostedGPT running the GPT-4o announced today!
I just upgraded the HostedGPT app to use the new GPT-4o model that just got released. I've been playing with it and it's INCREDIBLY fast; much faster than GPT-4. It's also a bit smarter too. It's winning on the Overall ELO rankings.
If you don't want to wait, now is a good time to set up your own chat interface. HostedGPT is incredibly easy to set up and it's a great user experience. This is the open source rails clone of ChatGPT. Demo video on this page too: https://github.com/allyourbot/hostedgpt
You just click the link, scroll down and click the purple button "Deploy to Render". After a few minutes, you will have your own deployed version. And because it's in rails, you can easily contribute to it.
In addition to getting the newest models right away, you'll never again get those annoying notices that you've used up your GPT-4 quota and need to come back in a few hours.
]]]
They might tell us the topics of the post are: HostedGPT, GPT-4o, ELO ranking, ChatGPT, open source, rails, Github, quota, OpenAI
It is okay, even desirable, for the topics to include topics which were not exactly mentioned in the post. but which the user would likely associate with the post.
Then the user might give the summary: Post announcing a new version of HostedGPT running GPT-4o. The post explains how to set it up and why it's better.
Always return this as JSON in the following format:
{ keywords: [ "...", "..." ], summary: "" }
I didn't write out the JSON but we should actually write out the JSON
From this Discussion #450
I think the key part of making vector search work well is to consistently generate a snippet for storage and retrieval. For example, when you came across this HostedGPT post and you bookmark it, there is some process of committing this to memory, maybe we call
Memorize.new.commit_link(url). In analyzing the link we construct a string such as:This would be the string we turn into a vector. The key idea is that this string is generic by following a universal heuristic. The heuristic I just used is:
Note: The two hard parts about this encoding are the post_summary and the post_keywords. When we are committing something to memory, we will need to come up with special prompt that generates these. (I'll include an example below)
Then, after this has been committed to memory, the user may ask for retrieval in a variety of ways. For example, they might ask the AI, "What was that ChatGPT clone built on Rails that I found on Reddit?" and they also might ask, "I saw a project last week on one of the platforms, I forget which one, it was an open source chat bot. Where did I read about that?" We don't use those exact strings for vector matching, instead we use an LLM to convert this question into our "snippet" format. The reason we're mapping things to a universal snippet is to help account for such differences in input text when they're essentially asking for the same question. To continue these examples:
When we are committing things, this is an example of a prompt which might generate the keywords and summary that we want:
I didn't write out the JSON but we should actually write out the JSON