[LDB-21] Neither Java nor Fennel calc handles arithmetic overflow

[reporter="jpham", created="Fri, 3 Feb 2006 15:29:21 -0500 (GMT-05:00)"] For example, take the following integer addition

0: jdbc:farrago:> values 2100000000 + 2100000000;
+------------+
| EXPR$0 |
+------------+
| -94967296 |
+------------+

Make the numbers a little larger (so they are bigint) and things work

0: jdbc:farrago:> values 3100000000 + 3100000000;
+-------------+
| EXPR$0 |
+-------------+
| 6200000000 |
+-------------+

But you still have problems

0: jdbc:farrago:> values 8876543210987654321 + 1000000000000000000;
+-----------------------+
|EXPR$0 |
+-----------------------+
| -8570200862721897295 |
+-----------------------+

I think the Fennel representation depends on the Boost type / arithmetic libraries while the generated Java code doesn't work because it checks the result too late.

On the positive side, both Java and Fennel calculators seem to overflow consistently. :)

0: jdbc:farrago:> alter system set "calcVirtualMachine"='CALCVM_JAVA';
No rows affected (0.123 seconds)
0: jdbc:farrago:> values 8876543210987654321 + 1000000000000000000;
+-----------------------+
|EXPR$0 |
+-----------------------+
| -8570200862721897295 |
+-----------------------+
1 row selected (0.118 seconds)
0: jdbc:farrago:> alter system set "calcVirtualMachine"='CALCVM_FENNEL';
No rows affected (0.055 seconds)
0: jdbc:farrago:> values 8876543210987654321 + 1000000000000000000;
+-----------------------+
|EXPR$0 |
+-----------------------+
| -8570200862721897295 |
+-----------------------+
1 row selected (0.952 seconds)

LucidDB / luciddb

[LDB-21] Neither Java nor Fennel calc handles arithmetic overflow #300