prestodb / presto

The official home of the Presto distributed SQL query engine for big data
http://prestodb.io
Apache License 2.0
16.06k stars 5.38k forks source link

round(x, d) returns wrong results when x*(10^d) is outside of Long range #23329

Closed zacw7 closed 3 months ago

zacw7 commented 3 months ago

Math function round(x, d) returns wrong results when x*(10^d) doesn't belong to [Long.MIN_VALUE, Long.MAX_VALUE]. For those input, the result will be Long.MIN_VALUE/d or Long.MAX_VALUE/d.

This is because the output of Math.round() is capped at Long.MIN_VALUE (for negative) and Long.MAX_VALUE (for positive). https://github.com/prestodb/presto/blob/db359e340efb620c618e385135c859aa3429ceac/presto-main/src/main/java/com/facebook/presto/operator/scalar/MathFunctions.java#L1103-L1108

Expected Behavior

SELECT ROUND(9223372036854775900.0) AS rounded; -- 9223372036854775900.0
SELECT ROUND(3574559470676000000.0, 1) AS rounded; -- 3574559470676000000.0
SELECT ROUND(-3574559470676000000.0, 1) AS rounded; -- -3574559470676000000.0
SELECT ROUND(92233720368547759.0, 100) AS rounded; -- 92233720368547759.0
SELECT ROUND(35745594706760000.0, 100) AS rounded; -- 35745594706760000.0
SELECT ROUND(-35745594706760000.0, 100) AS rounded; -- -35745594706760000.0

Current Behavior

SELECT ROUND(9223372036854775900.0) AS rounded; -- 922337203685480000.0
SELECT ROUND(3574559470676000000.0, 1) AS rounded; -- 922337203685480000.0
SELECT ROUND(-3574559470676000000.0, 1) AS rounded; -- -922337203685480000.0
SELECT ROUND(92233720368547759.0, 100) AS rounded; -- 9.2233720368548E-82
SELECT ROUND(35745594706760000.0, 100) AS rounded; -- 9.2233720368548E-82
SELECT ROUND(-35745594706760000.0, 100) AS rounded; -- -9.2233720368548E-82

Possible Solution

If x * d is larger than Long.MAX_VALUE or smaller than Long.MIN_VALUE, convert it into BigDecimal to ensure an accurate result.

Steps to Reproduce

Test case in https://github.com/prestodb/presto/pull/23330

Context

In Velox, it behaves properly.

amitkdutta commented 3 months ago

CC: @rschlussel