Open RogerThomas opened 10 months ago
I am trying to use the tags feature along with KNN vector search, so for example, a query searching for all albums where
(Genre:rock AND Decade: 60s) OR (Genre:rock AND Decade: 70s)
Along with a KNN vector search.
Querying on just the expression above is fine, however when I combine this with a KNN search, I get the following error:
redis.exceptions.ResponseError: Syntax error at offset 84 near >[
import numpy as np from redis_om import Field, JsonModel, VectorFieldOptions, Migrator, KNNExpression class Album(JsonModel): title: str = Field(primary_key=True) tags: str = Field(index=True) title_embeddings: list[float] = Field( index=True, vector_options=VectorFieldOptions.flat( type=VectorFieldOptions.TYPE.FLOAT32, dimension=2, distance_metric=VectorFieldOptions.DISTANCE_METRIC.COSINE, ), ) def embedding_function(text: str) -> [float]: """Mock embedding fnction, just counts vowels/consonants and normalises""" vowels = "aeiouAEIOU" num_vowels = sum((char in vowels) for char in text) num_consonants = len(text) - num_vowels x = num_vowels / len(text) y = num_consonants / len(text) norm = (x * x + y * y) ** 0.5 xdim = x / norm ydim = y / norm vector = [round(xdim, 9), round(ydim, 9)] return vector def main(): Migrator().run() album_tuples = [ ("The Dark Side Of The Moon", ["Genre:prog-rock", "Decade:70s"]), ("Rumours", ["Genre:rock", "Decade:70s"]), ("Abbey Road", ["Genre:rock", "Decade:60s"]), ] for title, tags in album_tuples: album = Album( title=title, tags="|".join(tags), title_embeddings=embedding_function(title), ) album.save() expression = (Album.tags == "Genre:rock|Decade:70s") | (Album.tags == "Genre:rock|Decade:60s") knn_expression = KNNExpression( k=3, vector_field=Album.__fields__["title_embeddings"], reference_vector=np.array(embedding_function("Hotel California"), dtype="f4").tobytes() ) # --------------------------------------------------------------------------------------------- # Query 1, Query on Tags only # --------------------------------------------------------------------------------------------- print("\nRunning Query 1, Tags only") albums = Album.find(expression).all() for album in albums: print(album) # --------------------------------------------------------------------------------------------- # Query 2, Query on KNN only # --------------------------------------------------------------------------------------------- print("\nRunning Query 2, KNN only") albums = Album.find(knn=knn_expression).all() for album in albums: print(album) # --------------------------------------------------------------------------------------------- # Query 2, Query on Tags and KNN # --------------------------------------------------------------------------------------------- print("\nRunning Query 2, KNN only") albums = Album.find(expression, knn=knn_expression).all() for album in albums: print(album) if __name__ == "__main__": main()
This will fail on this line
albums = Album.find(expression, knn=knn_expression).all()
I am trying to use the tags feature along with KNN vector search, so for example, a query searching for all albums where
Along with a KNN vector search.
Querying on just the expression above is fine, however when I combine this with a KNN search, I get the following error:
Code to reproduce
This will fail on this line