Closed zacw7 closed 3 months ago
Math function round(x, d) returns wrong results when x*(10^d) doesn't belong to [Long.MIN_VALUE, Long.MAX_VALUE]. For those input, the result will be Long.MIN_VALUE/d or Long.MAX_VALUE/d.
round(x, d)
x*(10^d)
Long.MIN_VALUE/d
Long.MAX_VALUE/d
This is because the output of Math.round() is capped at Long.MIN_VALUE (for negative) and Long.MAX_VALUE (for positive). https://github.com/prestodb/presto/blob/db359e340efb620c618e385135c859aa3429ceac/presto-main/src/main/java/com/facebook/presto/operator/scalar/MathFunctions.java#L1103-L1108
Long.MIN_VALUE
Long.MAX_VALUE
SELECT ROUND(9223372036854775900.0) AS rounded; -- 9223372036854775900.0 SELECT ROUND(3574559470676000000.0, 1) AS rounded; -- 3574559470676000000.0 SELECT ROUND(-3574559470676000000.0, 1) AS rounded; -- -3574559470676000000.0 SELECT ROUND(92233720368547759.0, 100) AS rounded; -- 92233720368547759.0 SELECT ROUND(35745594706760000.0, 100) AS rounded; -- 35745594706760000.0 SELECT ROUND(-35745594706760000.0, 100) AS rounded; -- -35745594706760000.0
SELECT ROUND(9223372036854775900.0) AS rounded; -- 922337203685480000.0 SELECT ROUND(3574559470676000000.0, 1) AS rounded; -- 922337203685480000.0 SELECT ROUND(-3574559470676000000.0, 1) AS rounded; -- -922337203685480000.0 SELECT ROUND(92233720368547759.0, 100) AS rounded; -- 9.2233720368548E-82 SELECT ROUND(35745594706760000.0, 100) AS rounded; -- 9.2233720368548E-82 SELECT ROUND(-35745594706760000.0, 100) AS rounded; -- -9.2233720368548E-82
If x * d is larger than Long.MAX_VALUE or smaller than Long.MIN_VALUE, convert it into BigDecimal to ensure an accurate result.
x * d
Test case in https://github.com/prestodb/presto/pull/23330
In Velox, it behaves properly.
CC: @rschlussel
Math function
round(x, d)
returns wrong results whenx*(10^d)
doesn't belong to [Long.MIN_VALUE, Long.MAX_VALUE]. For those input, the result will beLong.MIN_VALUE/d
orLong.MAX_VALUE/d
.This is because the output of Math.round() is capped at
Long.MIN_VALUE
(for negative) andLong.MAX_VALUE
(for positive). https://github.com/prestodb/presto/blob/db359e340efb620c618e385135c859aa3429ceac/presto-main/src/main/java/com/facebook/presto/operator/scalar/MathFunctions.java#L1103-L1108Expected Behavior
Current Behavior
Possible Solution
If
x * d
is larger than Long.MAX_VALUE or smaller than Long.MIN_VALUE, convert it into BigDecimal to ensure an accurate result.Steps to Reproduce
Test case in https://github.com/prestodb/presto/pull/23330
Context
In Velox, it behaves properly.