LucidDB / luciddb

DEFUNCT: See README
https://github.com/LucidDB/luciddb
Apache License 2.0
52 stars 24 forks source link

[FRG-10] Mismatch in Java and Fennel Calc when casting -1.5 to integer #860

Closed dynamobi-build closed 12 years ago

dynamobi-build commented 12 years ago

[reporter="angel", created="Mon, 26 Dec 2005 12:17:50 -0500 (GMT-05:00)"] Using fennel calc: cast(-1.5e0 as integer) gives -2
Using java calc: cast(-1.5e0 as integer) gives -1

Note: I'm doing cast(-1.5e0 as integer) to make sure that -1.5
is intepreted as a double instead of decimal.

See sample output using fennel and java calcs below:
0: jdbc:farrago:> alter system set "calcVirtualMachine" = 'CALCVM_FENNEL';
No rows affected (0.008 seconds)
0: jdbc:farrago:> values cast (1.75e0 as integer);
+---------+
| EXPR$0 |
+---------+
| 2 |
+---------+
1 row selected (0.324 seconds)
0: jdbc:farrago:> values cast (1.5e0 as integer);
+---------+
| EXPR$0 |
+---------+
| 2 |
+---------+
1 row selected (0.227 seconds)
0: jdbc:farrago:> values cast (-1.5e0 as integer);
+---------+
| EXPR$0 |
+---------+
| -2 |
+---------+
1 row selected (0.232 seconds)
0: jdbc:farrago:> values cast (-1.75e0 as integer);
+---------+
| EXPR$0 |
+---------+
| -2 |
+---------+
1 row selected (0.242 seconds)




0: jdbc:farrago:> alter system set "calcVirtualMachine" = 'CALCVM_JAVA';
No rows affected (0.043 seconds)
0: jdbc:farrago:> values cast (1.75e0 as integer);
+---------+
| EXPR$0 |
+---------+
| 2 |
+---------+
1 row selected (0.075 seconds)
0: jdbc:farrago:> values cast (1.5e0 as integer);
+---------+
| EXPR$0 |
+---------+
| 2 |
+---------+
1 row selected (0.08 seconds)
0: jdbc:farrago:> values cast (-1.5e0 as integer);
+---------+
| EXPR$0 |
+---------+
| -1 |
+---------+
1 row selected (0.083 seconds)
0: jdbc:farrago:> values cast (-1.75e0 as integer);
+---------+
| EXPR$0 |
+---------+
| -2 |
+---------+
1 row selected (0.067 seconds)

dynamobi-build commented 12 years ago

[author="angel", created="Mon, 26 Dec 2005 21:00:31 -0500 (GMT-05:00)"] Test case added to SqlOperatorTests - look for issueFrg10Fixed.

dynamobi-build commented 12 years ago

[author="xluo", created="Sun, 8 Jan 2006 13:12:33 -0500 (GMT-05:00)"] wrote new code to check if the number is negative or not.