JoinTheAlliance / bgent

Flexible, scalable and customizable agents to do your bidding.
https://bgent.org
69 stars 15 forks source link

Embedding caching #19

Closed lalalune closed 3 months ago

lalalune commented 4 months ago

Embeddings cost money, and 20-30% of them are probably repeats of obvious things, like "hey whats up" (in the case of a memory embedding).

When inserting embeddings, do a check to see if there are any rows where the string is the same. Bonus points for levenstein distance check to string similarity. Then if that row returns and has a vector, use it instead of making a new vector.

lalalune commented 3 months ago

@sshivaditya2019 I've merged this but we will need to ignore levenstein distance check if content is more than 255 characters

lalalune commented 3 months ago

I updated the levenshtein distance check on the server but we'll need to add a migration. I'm going to close this out and assume we'll handle migration on discord