apache / datafusion

Apache DataFusion SQL Query Engine
https://datafusion.apache.org/
Apache License 2.0
6.37k stars 1.2k forks source link

Decimal conversion inconsistency #7661

Open comphead opened 1 year ago

comphead commented 1 year ago

Describe the bug

A decimal conversion query is not consistent with PG or Spark

To Reproduce

❯ select cast(1.1 as decimal(2, 2)) + 1;
+-------------------------+
| Float64(1.1) + Int64(1) |
+-------------------------+
| 2.10                    |
+-------------------------+
1 row in set. Query took 0.002 seconds.

the same query returns null in Spark in non ansi mode numeric field overflow error in PG

Expected behavior

Should be consistent

Additional context

No response

comphead commented 1 year ago

cc @viirya

viirya commented 1 year ago

This is because cast kernel at upstream doesn't check precision overflow, although it checks casting overflow. I've submitted a change at the upstream for this: https://github.com/apache/arrow-rs/pull/4866

eejbyfeldt commented 3 weeks ago

On main this now produces

> select cast(1.1 as decimal(2, 2)) + 1;
Arrow error: Invalid argument error: 110 is too large to store in a Decimal128 of precision 2. Max is 9

@comphead should we close this issue?