quarkiverse / quarkus-langchain4j

Quarkus Langchain4j extension
https://docs.quarkiverse.io/quarkus-langchain4j/dev/index.html
Apache License 2.0
119 stars 65 forks source link

Implement embedding filtering where appropriate #368

Open jmartisk opened 3 months ago

jmartisk commented 3 months ago

LangChain4j 0.28 has a new metadata filter API for embedding stores. We should implement that for embedding stores where it is appropriate - for some embedding stores we don't just take the upstream implementation, we have our own instead.

Tracking:

humcqc commented 3 months ago

I can contribute on the PgVector part. Any advise to start ? Here is the example i'm thinking to start : https://github.com/langchain4j/langchain4j-examples/blob/main/rag-examples/src/main/java/_06_Metadata_Filtering.java#L100

jmartisk commented 3 months ago

You'll need to override the EmbeddingStore.search method in PgVectorEmbeddingStore in a way that it doesn't ignore the passed filter, but instead transforms the filter into a suitable WHERE SQL clause and uses that clause when retrieving embeddings from the database...

humcqc commented 3 months ago

First try -> https://github.com/quarkiverse/quarkus-langchain4j/pull/410