UnstructuredFilterTableRequest does not properly stringify some filter types

abaranec commented 3 months ago

Description

If you provide a QST object with a filter to a barrage subscription that mixes a structured filter with RawString filters, the BatchTableRequestBuilder does not properly stringify some of the filters. In particular in Strings.java

public static String of(FilterIn in) {
        return in.toString();
    }

but FilterIn does not implement toString, so you just get an invalid string expression and the subscription fails.

Steps to reproduce You will need two Core instances running. In one create a table with at least two columns (I used a string and a double column)

// Generate a random source table
strs=["Apple", "Banana", "Carrot", "Dalmation", "Elephant", "Frog", "Guppy"]
random = new Random(1)
table=timeTableBuilder().period("PT0.1S").build()
    .update("Sequence=ii", 
            "StrVal=(String)strs.get(i%strs.size())", 
            "DVal=random.nextDouble()")

In the other instance, create a Barrage session to connect to the first then fetch the table using a QST object

TableSpec tableSpec = TicketTable.fromQueryScopeField("table")
    tableSpec.where(Filter.and(
        FilterIn.of(ColumnName.of("StrVal"), Literal.of("Apple")),
        RawString.of("DVal > .5"),
        RawString.of("Dval < .9")
    );

sub = session.getSession().subscribe(
                        tableSpec,
                        SUB_OPTIONS);

Expected results

A table is returned that applies the desired filters

Actual results

Internal Error '56b2c513-834b-4b15-87f0-9913b5b62bcc' io.deephaven.engine.table.impl.select.FormulaCompilationException: Formula compilation error for: (FilterIn{expression=ColumnName(StrVal), values=[LiteralString{value=Apple}]})
    at io.deephaven.engine.table.impl.select.DhFormulaColumn.initDef(DhFormulaColumn.java:216)
    at io.deephaven.engine.table.impl.select.SwitchColumn.initDef(SwitchColumn.java:64)
    at io.deephaven.engine.table.impl.select.analyzers.SelectAndViewAnalyzer.create(SelectAndViewAnalyzer.java:124)
    at io.deephaven.engine.table.impl.select.analyzers.SelectAndViewAnalyzer.create(SelectAndViewAnalyzer.java:73)
    at io.deephaven.engine.table.impl.QueryTable.validateSelect(QueryTable.java:1490)
    at io.deephaven.server.table.validation.ColumnExpressionValidator.validateColumnExpressions(ColumnExpressionValidator.java:106)
    at io.deephaven.server.table.validation.ColumnExpressionValidator.validateSelectFilters(ColumnExpressionValidator.java:95)
    at io.deephaven.server.table.ops.UnstructuredFilterTableGrpcImpl.create(UnstructuredFilterTableGrpcImpl.java:37)
    at io.deephaven.server.table.ops.UnstructuredFilterTableGrpcImpl.create(UnstructuredFilterTableGrpcImpl.java:21)
    at io.deephaven.server.table.ops.TableServiceGrpcImpl$BatchExportBuilder.doExport(TableServiceGrpcImpl.java:757)
    at io.deephaven.server.session.SessionState$ExportObject.doExport(SessionState.java:995)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at io.deephaven.server.runner.scheduler.SchedulerModule$ThreadFactory.lambda$newThread$0(SchedulerModule.java:100)
    at java.base/java.lang.Thread.run(Thread.java:840)

Additional details and attachments

If applicable, add any additional screenshots, logs, or other attachments to help explain your problem.

Versions

Deephaven: 0.34.3

rcaudy commented 3 months ago

I think this is also wrong:

    public static String of(FilterIn in, boolean invert) {
        final String inner = of(in);
        return (invert ? "!" : "") + inner;
    }

rcaudy commented 3 months ago

This Strings.of needs to be the inverse of io.deephaven.engine.table.impl.select.WhereFilterFactory#getExpression. That is not the case for a number of filter types right now. We should blow up for the unsupported ones, and have correct Strings.of for the rest.

devinrsmith commented 3 months ago

deephaven / deephaven-core

UnstructuredFilterTableRequest does not properly stringify some filter types #5665