search

langchain-ai / langchain-postgres

LangChain abstractions backed by Postgres Backend
MIT License
133 stars 48 forks source link

Nested Metadata filtering broken #56

Open NPuhlmann opened 6 months ago

NPuhlmann commented 6 months ago

I used to filter metadata with the similiarity_search_with_relevance_scores method. The filter I use is: {'file_metadata': {'modified': {'$between': [datetime.date(2022, 3, 21), datetime.date(2024, 5, 21)]}}, 'source': 'internet'}.

I would expect the filter to be working, since it reflects my metadata in postgres which has following format: "source": "xyz", "content": null, "file_url": "https://xyz.pdf", "filename": "xyz.pdf", "mime_type": "mime_type", "file_metadata": { "title": "Test", "author": "Santa Clause", "created": "2024-04-23T08:38:54", "subject": "", "version": "", "category": "", "comments": "", "keywords": "", "language": "", "modified": "2024-04-23T08:38:54", "revision": 188, "identifier": "", "last_printed": "", "content_status": "", "last_modified_by": "elv" }, ....

That was working before, however now i get the error ValueError: Invalid operator: modified. Expected one of {'$between', '$ilike', '$lt', '$gt', '$gte', '$lte', '$ne', '$or', '$and', '$exists', '$in', '$eq', '$like', '$nin'}

Am I missing something on how to create the filters or is the feature broken?

Current Versions: langchain~=0.1.16 langchain-community~=0.0.36 langchain-core~=0.1.52 langchain-openai~=0.0.8 langchain-postgres~=0.0.6

eyurtsev commented 5 months ago

Don't think this supports nested metadata right now. Someone can pick that up as a feature to add.

I don't know whether the above syntax makes sense or whether it would need to be modified to use a List[str] for specifying paths. The main question is whether the syntax is self-consistent.