search

apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.26k stars 1.23k forks source link

Issues with min/max over big_decimal field when using Multi-stage engine #12705

Open dorlevi opened 3 months ago

dorlevi commented 3 months ago

A simple min/max over a big_decimal fails when using the multi-stage engine,

Query: select min(block_number) from org_2dYiMRMfas142XRKQ3bJIqmN3V6_erc20_balance_changes limit 10

Error:

Error Code: 200

QueryExecutionError:
Received error query execution result block: {1000=Unable to create DataBlock
java.lang.RuntimeException: Unable to create DataBlock
    at org.apache.pinot.query.runtime.blocks.TransferableBlock.getDataBlock(TransferableBlock.java:140)
    at org.apache.pinot.query.mailbox.GrpcSendingMailbox.toMailboxContent(GrpcSendingMailbox.java:121)
    at org.apache.pinot.query.mailbox.GrpcSendingMailbox.send(GrpcSendingMailbox.java:72)
    at org.apache.pinot.query.runtime.operator.exchange.BlockExchange.sendBlock(BlockExchange.java:126)
...
Caused by: java.lang.ClassCastException: class java.lang.Double cannot be cast to class java.math.BigDecimal (java.lang.Double and java.math.BigDecimal are in module java.base of loader 'bootstrap')
    at org.apache.pinot.core.common.datablock.DataBlockBuilder.buildFromRows(DataBlockBuilder.java:148)
    at org.apache.pinot.query.runtime.blocks.TransferableBlock.getDataBlock(TransferableBlock.java:133)
    ... 16 more}
org.apache.pinot.query.service.dispatch.QueryDispatcher.getResultTable(QueryDispatcher.java:332)
org.apache.pinot.query.service.dispatch.QueryDispatcher.runReducer(QueryDispatcher.java:276)
org.apache.pinot.query.service.dispatch.QueryDispatcher.submitAndReduce(QueryDispatcher.java:100)
org.apache.pinot.broker.requesthandler.MultiStageBrokerRequestHandler.handleRequest(MultiStageBrokerRequestHandler.java:199)

the same query works on v1 query engine.

Jackie-Jiang commented 3 months ago

Could be related to #12670

Jackie-Jiang commented 3 months ago

Can you try select min(cast(block_number as double)) from org_2dYiMRMfas142XRKQ3bJIqmN3V6_erc20_balance_changes limit 10 and see if it works? Even in v1 this query won't be executed on big_decimal, but on double with potential precision loss

dorlevi commented 3 months ago

The original queryselect min(block_number) from org_2dYiMRMfas142XRKQ3bJIqmN3V6_erc20_balance_changes limit 10 works well on v1 for us; tried your version casting as double works. but it shouldn't be necessary here

Jackie-Jiang commented 3 months ago

v2 is strong typed, and currently min() takes double as input and return double, that is why it failed in v2. Ideally min() should be able to directly take big_decimal and return big_decimal.

dorlevi commented 3 months ago

Ideally it would support all the types that other calcite compatible DBs would accept