search

apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.42k stars 1.27k forks source link

Filtering on a boolean field with just the field name results in NullPointerException #8444

Closed jadami10 closed 2 years ago

jadami10 commented 2 years ago

Running a query with just select <fields> from <table> where <boolean_field> results in a null point exception. This should be a supported query.

Stack trace from pinot

[2022-03-30 20:40:05.343615] java.lang.NullPointerException: null
[2022-03-30 20:40:05.343660]    at org.apache.pinot.core.query.optimizer.filter.FlattenAndOrFilterOptimizer.optimize(FlattenAndOrFilterOptimizer.java:69) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343694]    at org.apache.pinot.core.query.optimizer.filter.FlattenAndOrFilterOptimizer.optimize(FlattenAndOrFilterOptimizer.java:77) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343736]    at org.apache.pinot.core.query.optimizer.filter.FlattenAndOrFilterOptimizer.optimize(FlattenAndOrFilterOptimizer.java:64) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343766]    at org.apache.pinot.core.query.optimizer.QueryOptimizer.optimize(QueryOptimizer.java:81) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343796]    at org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleSQLRequest(BaseBrokerRequestHandler.java:404) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343830]    at org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleRequest(BaseBrokerRequestHandler.java:196) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343863]    at org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleRequest(BaseBrokerRequestHandler.java:101) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343892]    at org.apache.pinot.broker.api.resources.PinotClientRequest.processSqlQueryPost(PinotClientRequest.java:191) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343915]    at jdk.internal.reflect.GeneratedMethodAccessor64.invoke(Unknown Source) ~[?:?]
[2022-03-30 20:40:05.343938]    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
[2022-03-30 20:40:05.343963]    at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
[2022-03-30 20:40:05.344004]    at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344039]    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344075]    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344105]    at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:159) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344140]    at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344177]    at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344222]    at org.glassfish.jersey.server.model.ResourceMethodInvoker.lambda$apply$0(ResourceMethodInvoker.java:381) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344257]    at org.glassfish.jersey.server.ServerRuntime$AsyncResponder$2$1.run(ServerRuntime.java:819) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344289]    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344312]    at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344346]    at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344372]    at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344398]    at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344429]    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344456]    at org.glassfish.jersey.server.ServerRuntime$AsyncResponder$2.run(ServerRuntime.java:814) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344480]    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
[2022-03-30 20:40:05.344524]    at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
[2022-03-30 20:40:05.344545]    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
[2022-03-30 20:40:05.344566]    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
[2022-03-30 20:40:05.344582]    at java.lang.Thread.run(Thread.java:829) [?:?]

user facing error

ProcessingException(errorCode:450, message:InternalError:
java.io.IOException: Failed : HTTP error code : 500
    at org.apache.pinot.controller.api.resources.PinotQueryResource.sendPostRaw(PinotQueryResource.java:305)
    at org.apache.pinot.controller.api.resources.PinotQueryResource.sendRequestRaw(PinotQueryResource.java:343)
    at org.apache.pinot.controller.api.resources.PinotQueryResource.getQueryResponse(PinotQueryResource.java:225)
    at org.apache.pinot.controller.api.resources.PinotQueryResource.handlePostSql(PinotQueryResource.java:138))
KKcorps commented 2 years ago

@Jackie-Jiang Should we go with re-writing the query WHERE A to WHERE A IS TRUE ?

Jackie-Jiang commented 2 years ago

@KKcorps Good solution! We can re-write the filter expression if the underlying predicate does not have FilterKind as the function. Pinot supports WHERE A = true

siddharthteotia commented 2 years ago

@Jackie-Jiang / @KKcorps / @jadami10 are you planning to work on this ?

If not, @vvivekiyer can help pick it up as he is familiarizing himself through the code

KKcorps commented 2 years ago

@siddharthteotia I am working on BrokerResponse so @vvivekiyer can pick it up

siddharthteotia commented 2 years ago

@vvivekiyer is working on the fix

siddharthteotia commented 2 years ago

Fixed with https://github.com/apache/pinot/pull/8518

jadami10 commented 2 years ago

thanks for working on this! I didn't see a test example, but does WHERE NOT <boolean_field> work correctly?

vvivekiyer commented 2 years ago

Hi Johan, yes it should. I just tested it out locally for a use-case. I didn't add a test-case example because AND, OR, NOT are processed similarly.