deephaven / deephaven-core

Deephaven Community Core
Other
242 stars 78 forks source link

RangeCondition not used when filter string comparison contains `!` #3738

Open devinrsmith opened 1 year ago

devinrsmith commented 1 year ago

Inverting a filter expressing string with a String column and a char literal comparison does not work.

from deephaven.column import string_col

t = new_table([string_col("X", ["a", "b", "c", None])])

t1 = t.where("X <= `b`")
t2 = t.where("X <= 'b'")
t3 = t.where("!(X <= `b`)")
t4 = t.where("!(X <= 'b')")

t1, t2, and t3 parse and execute as expected.

t4 fails with

java.lang.ClassCastException encountered in filter={ !(X <= 'b') }
    at io.deephaven.engine.table.impl.select.ConditionFilter$ChunkFilter.filter(ConditionFilter.java:360)
    at io.deephaven.engine.table.impl.select.AbstractConditionFilter.filter(AbstractConditionFilter.java:308)
    at io.deephaven.engine.table.impl.QueryTable.filterRows(QueryTable.java:1129)
    at io.deephaven.engine.table.impl.QueryTable.lambda$whereInternal$28(QueryTable.java:1061)
    at io.deephaven.engine.table.impl.BaseTable.initializeWithSnapshot(BaseTable.java:1215)
    at io.deephaven.engine.table.impl.QueryTable.lambda$whereInternal$29(QueryTable.java:1055)
    at io.deephaven.engine.table.impl.QueryTable.memoizeResult(QueryTable.java:3163)
    at io.deephaven.engine.table.impl.QueryTable.lambda$whereInternal$30(QueryTable.java:1050)
    at io.deephaven.engine.table.impl.perf.QueryPerformanceRecorder.withNugget(QueryPerformanceRecorder.java:519)
    at io.deephaven.engine.table.impl.QueryTable.whereInternal(QueryTable.java:1005)
    at io.deephaven.engine.table.impl.QueryTable.where(QueryTable.java:1001)
    at io.deephaven.engine.table.impl.QueryTable.where(QueryTable.java:104)
    at io.deephaven.api.TableOperationsDefaults.where(TableOperationsDefaults.java:51)
devinrsmith commented 1 year ago

Likely related to https://github.com/deephaven/deephaven-core/issues/3631

devinrsmith commented 1 year ago

This is mostly a ticket about the inconsistency of using ! in filter expression strings.

I'm betting this works in the non-inverted case b/c of special logic in matching the non-inverted case and creating a range condition filter:

        // <ColumnName> < <Number|Boolean|"String">
        // <ColumnName> <= <Number|Boolean|"String">
        // <ColumnName> > <Number|Boolean|"String">
        // <ColumnName> >= <Number|Boolean|"String">
        parser.registerFactory(new AbstractExpressionFactory<>(
        ...
                            return new RangeConditionFilter(columnName, conditionString, value, expression,
                            parserConfiguration);
                            ...