[Bug]: Text match filter doesn't work with PostgreSQL filter search and gives unknown operator error.

[BurnyCoder]

Bug Description

I have PostgreSQL db with Llamaindex and I am doing filter search over it. Many operators work but text match filter doesnt work. I have newest version of Llamaindex. List of filter operators including text match: https://docs.llamaindex.ai/en/v0.10.17/api/llama_index.core.vector_stores.FilterOperator.html

Version

llama-index-0.10.45

Steps to Reproduce

My code:

        filters = MetadataFilters(
            filters=[
                MetadataFilter(key="title", 
                operator=FilterOperator.TEXT_MATCH,        
                value="Graph RAG")
            ]
        )

Alternatively: operator="text_match" Rest of code is basically like https://docs.llamaindex.ai/en/stable/examples/vector_stores/postgres/

Relevant Logs/Tracbacks

It gives this error: 
 operator: FilterOperator.TEXT_MATCH, fallback to '='

[dosubot[bot]]

Hey @BurnyCoder! I'm here to help you with any bugs, questions, or contributions you have for the repository. Let's work together to resolve your issue.

The TEXT_MATCH filter operator is mapped to the SQL LIKE operator in the sql_operator_mapper dictionary. If the TEXT_MATCH filter operator is not being recognized and is falling back to the default = operator, it could be due to an issue in how the FilterOperator is being used or referenced in your code.

Ensure that the FilterOperator.TEXT_MATCH is correctly imported and used in your filter search logic. Here is the relevant code snippet:

from llama_index.core.vector_stores.types import (
    FilterOperator,
)

sql_operator_mapper = {
    FilterOperator.EQ: " = ",
    FilterOperator.GT: " > ",
    FilterOperator.GTE: " >= ",
    FilterOperator.LTE: " <= ",
    FilterOperator.TEXT_MATCH: " LIKE ",
    FilterOperator.NE: " NOT LIKE ",
    FilterOperator.IN: " IN ",
}

Additionally, the _to_postgres_operator method in the PGVectorStore class is responsible for converting FilterOperator values to their corresponding SQL operators. Ensure that this method includes a case for FilterOperator.TEXT_MATCH:

def _to_postgres_operator(self, operator: FilterOperator) -> str:
    if operator == FilterOperator.EQ:
        return "="
    elif operator == FilterOperator.GT:
        return ">"
    elif operator == FilterOperator.LT:
        return "<"
    elif operator == FilterOperator.NE:
        return "!="
    elif operator == FilterOperator.GTE:
        return ">="
    elif operator == FilterOperator.LTE:
        return "<="
    elif operator == FilterOperator.IN:
        return "IN"
    elif operator == FilterOperator.NIN:
        return "NOT IN"
    elif operator == FilterOperator.CONTAINS:
        return "@>"
    elif operator == FilterOperator.TEXT_MATCH:
        return "LIKE"
    else:
        _logger.warning(f"Unknown operator: {operator}, fallback to '='")
        return "="

Make sure that your filter search logic correctly references FilterOperator.TEXT_MATCH and uses the sql_operator_mapper dictionary to map it to the LIKE operator. If the issue persists, there might be a problem elsewhere in your code where the filter operators are being processed or applied.

Additionally, you can refer to the test cases provided in the llama-index-integrations/vector_stores/llama-index-vector-stores-postgres/tests/test_postgres.py file to ensure that your implementation aligns with the expected behavior. The test cases cover various scenarios, including the use of different filter operators, and can help you identify any discrepancies in your code ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[logan-markewich]

That filter type just isn't implemented right now, someone should have to make a PR to add that