LucidDB / luciddb

DEFUNCT: See README
https://github.com/LucidDB/luciddb
Apache License 2.0
53 stars 24 forks source link

[FRG-54] java calc does not always round properly when casting from one datatype to another #816

Closed dynamobi-build closed 12 years ago

dynamobi-build commented 12 years ago

[reporter="schoi", created="Thu, 23 Feb 2006 16:23:51 -0500 (GMT-05:00)"] Repro:
-----------
-- using JAVA calc
alter system set "calcVirtualMachine"='CALCVM_JAVA';

create table iT (i1 integer primary key, i integer);

insert into iT values(1, cast (1.99 as integer));
insert into iT values(2, cast (1.99 as double));
insert into iT values(3, cast (1.99 as float));
insert into iT values(4, cast (1.99 as real));
insert into iT values(5, cast (1.99 as decimal));

select i from iT;
+----+
| I |
+----+
| 2 |
| 1 |
| 1 |
| 1 |
| 2 |
+----+

-- Using FENNEL calc, re-run the same thing..

alter system set "calcVirtualMachine"='CALCVM_FENNEL';
...
select i from iT;
+----+
| I |
+----+
| 2 |
| 2 |
| 2 |
| 2 |
| 2 |
+----+

dynamobi-build commented 12 years ago

[author="jpham", created="Mon, 17 Apr 2006 17:40:37 -0500 (GMT-05:00)"] oops.. i just noticed this one. it is probably related

dynamobi-build commented 12 years ago

[author="elin", created="Thu, 21 Sep 2006 17:20:26 -0500 (GMT-05:00)"] Perhaps this, FRG-171 and FRG-204 are related?
luciddb test numerics.decimal illustrates this

dynamobi-build commented 12 years ago

[author="jpham", created="Sun, 24 Sep 2006 14:47:31 -0500 (GMT-05:00)"] Should be fixed by lu 7760

dynamobi-build commented 12 years ago

[author="jpham", created="Sun, 24 Sep 2006 14:57:03 -0500 (GMT-05:00)"] duplicate of FRG-204