search

br1ghtyang / asterixdb

Automatically exported from code.google.com/p/asterixdb
0 stars 0 forks source link

TPCH test fails due to difference in precision of value returned by SUM aggregate function #64

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
Hi Yingyi,

Test tpch/q9product_type_profit_nt.aql, fails due to difference in precision of 
value returned by SUM aggregate function.

Difference in results is as follows,

< { "nation": "IRAN", "o_year": 1997, "sum_profit": 52643.77359999999d }
> { "nation": "IRAN", "o_year": 1997, "sum_profit": 52643.7736d }

Should I modify the value returned by sum agg. function in the expected result 
file or is this a bug ?

Asterix build revision : 47
Hyracks_dev_next revision : 1306
Test platform : Ubuntu 64 bit
--------------------------------------------------------------------------------
-------------------------

Failing test case  : tpch/q9product_type_profit_nt.aql

Failing query is as follows,
...
 let $amount := $l3.l_extendedprice * (1 - $l3.l_discount) -  $l3.ps_supplycost * $l3.l_quantity 
...
return 
   { "nation": $nation, 
     "o_year": $o_year, 
      "sum_profit": sum( for $pr in $profit return $pr.amount )  }    

Here is the stack trace

java.lang.AssertionError: Result for 
src/test/resources/runtimets/queries/tpch/q9_product_type_profit_nt.aql changed 
at line 13:
< { "nation": "IRAN", "o_year": 1997, "sum_profit": 52643.77359999999d }
> { "nation": "IRAN", "o_year": 1997, "sum_profit": 52643.7736d }
    at org.junit.Assert.fail(Assert.java:91)
    at edu.uci.ics.asterix.test.aql.TestsUtils.runScriptAndCompareWithResult(TestsUtils.java:75)
    at edu.uci.ics.asterix.test.runtime.RuntimeTest.test(RuntimeTest.java:154)
    at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
    at org.junit.runners.Suite.runChild(Suite.java:128)
    at org.junit.runners.Suite.runChild(Suite.java:24)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)

--------------------------------------------------------------------------------
----------------------------

----------Logical plan:
write [%0->$$91] -- |UNPARTITIONED|
  project ([$$91]) -- |UNPARTITIONED|
    assign [$$91] <- [function-call: asterix:open-record-constructor, Args:[AString: {nation}, %0->$$13, AString: {o_year}, %0->$$14, AString: {sum_profit}, function-call: asterix:sum, Args:[%0->$$94]]] -- |UNPARTITIONED|
      subplan {
                aggregate [$$94] <- [function-call: asterix:listify, Args:[%0->$$93]] -- |UNPARTITIONED|
                  assign [$$93] <- [function-call: asterix:field-access-by-name, Args:[%0->$$16, AString: {amount}]] -- |UNPARTITIONED|
                    unnest $$16 <- function-call: asterix:scan-collection, Args:[%0->$$90] -- |UNPARTITIONED|
                      nested tuple source -- |UNPARTITIONED|
             } -- |UNPARTITIONED|
        order (ASC, %0->$$13) (DESC, %0->$$14)  -- |UNPARTITIONED|
          group by ([$$13 := function-call: asterix:field-access-by-name, Args:[%0->$$15, AString: {nation}]; $$14 := function-call: asterix:field-access-by-name, Args:[%0->$$15, AString: {o_year}]]) decor ([]) {
                    aggregate [$$90] <- [function-call: asterix:listify, Args:[%0->$$15]] -- |UNPARTITIONED|
                      nested tuple source -- |UNPARTITIONED|
                 } -- |UNPARTITIONED|
            unnest $$15 <- function-call: asterix:scan-collection, Args:[%0->$$87] -- |UNPARTITIONED|
              subplan {
                        aggregate [$$87] <- [function-call: asterix:listify, Args:[%0->$$85]] -- |UNPARTITIONED|
                          assign [$$85] <- [function-call: asterix:open-record-constructor, Args:[AString: {nation}, function-call: asterix:field-access-by-name, Args:[%0->$$2, AString: {n_name}], AString: {o_year}, %0->$$12, AString: {amount}, %0->$$11]] -- |UNPARTITIONED|
                            assign [$$12] <- [function-call: asterix:year, Args:[function-call: asterix:field-access-by-name, Args:[%0->$$1, AString: {o_orderdate}]]] -- |UNPARTITIONED|
                              assign [$$11] <- [function-call: asterix:numeric-subtract, Args:[function-call: asterix:numeric-multiply, Args:[function-call: asterix:field-access-by-name, Args:[%0->$$2, AString: {l_extendedprice}], function-call: asterix:numeric-subtract, Args:[AInt32: {1}, function-call: asterix:field-access-by-name, Args:[%0->$$2, AString: {l_discount}]]], function-call: asterix:numeric-multiply, Args:[function-call: asterix:field-access-by-name, Args:[%0->$$2, AString: {ps_supplycost}], function-call: asterix:field-access-by-name, Args:[%0->$$2, AString: {l_quantity}]]]] -- |UNPARTITIONED|
                                select (function-call: algebricks:eq, Args:[function-call: asterix:field-access-by-name, Args:[%0->$$1, AString: {o_orderkey}], function-call: asterix:field-access-by-name, Args:[%0->$$2, AString: {l_orderkey}]]) -- |UNPARTITIONED|
                                  unnest $$2 <- function-call: asterix:scan-collection, Args:[%0->$$71] -- |UNPARTITIONED|
                                    subplan {
                                              aggregate [$$71] <- [function-call: asterix:listify, Args:[%0->$$64]] -- |UNPARTITIONED|
                                                assign [$$64] <- [function-call: asterix:open-record-constructor, Args:[AString: {l_extendedprice}, function-call: asterix:field-access-by-name, Args:[%0->$$4, AString: {l_extendedprice}], AString: {l_discount}, function-call: asterix:field-access-by-name, Args:[%0->$$4, AString: {l_discount}], AString: {l_quantity}, function-call: asterix:field-access-by-name, Args:[%0->$$4, AString: {l_quantity}], AString: {l_orderkey}, function-call: asterix:field-access-by-name, Args:[%0->$$4, AString: {l_orderkey}], AString: {n_name}, function-call: asterix:field-access-by-name, Args:[%0->$$4, AString: {n_name}], AString: {ps_supplycost}, function-call: asterix:field-access-by-name, Args:[%0->$$4, AString: {ps_supplycost}]]] -- |UNPARTITIONED|
                                                  select (function-call: algebricks:and, Args:[function-call: asterix:contains, Args:[function-call: asterix:field-access-by-name, Args:[%0->$$3, AString: {p_name}], AString: {green}], function-call: algebricks:eq, Args:[function-call: asterix:field-access-by-name, Args:[%0->$$3, AString: {p_partkey}], function-call: asterix:field-access-by-name, Args:[%0->$$4, AString: {l_partkey}]]]) -- |UNPARTITIONED|
                                                    unnest $$4 <- function-call: asterix:scan-collection, Args:[%0->$$57] -- |UNPARTITIONED|
                                                      subplan {
                                                                aggregate [$$57] <- [function-call: asterix:listify, Args:[%0->$$49]] -- |UNPARTITIONED|
                                                                  assign [$$49] <- [function-call: asterix:open-record-constructor, Args:[AString: {l_extendedprice}, function-call: asterix:field-access-by-name, Args:[%0->$$6, AString: {l_extendedprice}], AString: {l_discount}, function-call: asterix:field-access-by-name, Args:[%0->$$6, AString: {l_discount}], AString: {l_quantity}, function-call: asterix:field-access-by-name, Args:[%0->$$6, AString: {l_quantity}], AString: {l_partkey}, function-call: asterix:field-access-by-name, Args:[%0->$$6, AString: {l_partkey}], AString: {l_orderkey}, function-call: asterix:field-access-by-name, Args:[%0->$$6, AString: {l_orderkey}], AString: {n_name}, function-call: asterix:field-access-by-name, Args:[%0->$$6, AString: {n_name}], AString: {ps_supplycost}, function-call: asterix:field-access-by-name, Args:[%0->$$5, AString: {ps_supplycost}]]] -- |UNPARTITIONED|
                                                                    select (function-call: algebricks:and, Args:[function-call: algebricks:eq, Args:[function-call: asterix:field-access-by-name, Args:[%0->$$5, AString: {ps_suppkey}], function-call: asterix:field-access-by-name, Args:[%0->$$6, AString: {l_suppkey}]], function-call: algebricks:eq, Args:[function-call: asterix:field-access-by-name, Args:[%0->$$5, AString: {ps_partkey}], function-call: asterix:field-access-by-name, Args:[%0->$$6, AString: {l_partkey}]]]) -- |UNPARTITIONED|
                                                                      unnest $$6 <- function-call: asterix:scan-collection, Args:[%0->$$41] -- |UNPARTITIONED|
                                                                        subplan {
                                                                                  aggregate [$$41] <- [function-call: asterix:listify, Args:[%0->$$33]] -- |UNPARTITIONED|
                                                                                    assign [$$33] <- [function-call: asterix:open-record-constructor, Args:[AString: {l_suppkey}, function-call: asterix:field-access-by-name, Args:[%0->$$10, AString: {l_suppkey}], AString: {l_extendedprice}, function-call: asterix:field-access-by-name, Args:[%0->$$10, AString: {l_extendedprice}], AString: {l_discount}, function-call: asterix:field-access-by-name, Args:[%0->$$10, AString: {l_discount}], AString: {l_quantity}, function-call: asterix:field-access-by-name, Args:[%0->$$10, AString: {l_quantity}], AString: {l_partkey}, function-call: asterix:field-access-by-name, Args:[%0->$$10, AString: {l_partkey}], AString: {l_orderkey}, function-call: asterix:field-access-by-name, Args:[%0->$$10, AString: {l_orderkey}], AString: {n_name}, function-call: asterix:field-access-by-name, Args:[%0->$$7, AString: {n_name}]]] -- |UNPARTITIONED|
                                                                                      select (function-call: algebricks:eq, Args:[function-call: asterix:field-access-by-name, Args:[%0->$$7, AString: {s_suppkey}], function-call: asterix:field-access-by-name, Args:[%0->$$10, AString: {l_suppkey}]]) -- |UNPARTITIONED|
                                                                                        unnest $$10 <- function-call: asterix:dataset, Args:[AString: {LineItem}] -- |UNPARTITIONED|
                                                                                          unnest $$7 <- function-call: asterix:scan-collection, Args:[%0->$$28] -- |UNPARTITIONED|
                                                                                            subplan {
                                                                                                      aggregate [$$28] <- [function-call: asterix:listify, Args:[%0->$$25]] -- |UNPARTITIONED|
                                                                                                        assign [$$25] <- [function-call: asterix:open-record-constructor, Args:[AString: {s_suppkey}, function-call: asterix:field-access-by-name, Args:[%0->$$8, AString: {s_suppkey}], AString: {n_name}, function-call: asterix:field-access-by-name, Args:[%0->$$9, AString: {n_name}]]] -- |UNPARTITIONED|
                                                                                                          select (function-call: algebricks:eq, Args:[function-call: asterix:field-access-by-name, Args:[%0->$$9, AString: {n_nationkey}], function-call: asterix:field-access-by-name, Args:[%0->$$8, AString: {s_nationkey}]]) -- |UNPARTITIONED|
                                                                                                            unnest $$9 <- function-call: asterix:dataset, Args:[AString: {Nation}] -- |UNPARTITIONED|
                                                                                                              unnest $$8 <- function-call: asterix:dataset, Args:[AString: {Supplier}] -- |UNPARTITIONED|
                                                                                                                nested tuple source -- |UNPARTITIONED|
                                                                                                   } -- |UNPARTITIONED|
                                                                                              nested tuple source -- |UNPARTITIONED|
                                                                               } -- |UNPARTITIONED|
                                                                          unnest $$5 <- function-call: asterix:dataset, Args:[AString: {PartSupp}] -- |UNPARTITIONED|
                                                                            nested tuple source -- |UNPARTITIONED|
                                                             } -- |UNPARTITIONED|
                                                        unnest $$3 <- function-call: asterix:dataset, Args:[AString: {Part}] -- |UNPARTITIONED|
                                                          nested tuple source -- |UNPARTITIONED|
                                           } -- |UNPARTITIONED|
                                      unnest $$1 <- function-call: asterix:dataset, Args:[AString: {Orders}] -- |UNPARTITIONED|
                                        nested tuple source -- |UNPARTITIONED|
                     } -- |UNPARTITIONED|
                empty-tuple-source -- |UNPARTITIONED|
-- SINK_WRITE  |PARTITIONED|
  -- STREAM_PROJECT  |PARTITIONED|
    -- ASSIGN  |PARTITIONED|
      -- SORT_MERGE_EXCHANGE [$$13(ASC), $$14(DESC) ]  |PARTITIONED|
        -- PRE_CLUSTERED_GROUP_BY[$$148, $$149]  |PARTITIONED|
                {
                  -- AGGREGATE  |LOCAL|
                    -- NESTED_TUPLE_SOURCE  |LOCAL|
                }
          -- HASH_PARTITION_MERGE_EXCHANGE MERGE:[$$148(ASC), $$149(DESC)] HASH:[$$148, $$149]  |PARTITIONED|
            -- PRE_CLUSTERED_GROUP_BY[$$110, $$12]  |LOCAL|
                    {
                      -- AGGREGATE  |LOCAL|
                        -- NESTED_TUPLE_SOURCE  |LOCAL|
                    }
              -- ONE_TO_ONE_EXCHANGE  |LOCAL|
                -- STABLE_SORT [$$110(ASC), $$12(DESC)]  |LOCAL|
                  -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                    -- STREAM_PROJECT  |PARTITIONED|
                      -- ASSIGN  |PARTITIONED|
                        -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                          -- HYBRID_HASH_JOIN [$$97][$$103]  |PARTITIONED|
                            -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                              -- STREAM_PROJECT  |PARTITIONED|
                                -- ASSIGN  |PARTITIONED|
                                  -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                    -- DATASOURCE_SCAN  |PARTITIONED|
                                      -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                        -- EMPTY_TUPLE_SOURCE  |PARTITIONED|
                            -- HASH_PARTITION_EXCHANGE [$$103]  |PARTITIONED|
                              -- STREAM_PROJECT  |PARTITIONED|
                                -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                  -- HYBRID_HASH_JOIN [$$98][$$117]  |PARTITIONED|
                                    -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                      -- STREAM_PROJECT  |PARTITIONED|
                                        -- STREAM_SELECT  |PARTITIONED|
                                          -- ASSIGN  |PARTITIONED|
                                            -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                              -- DATASOURCE_SCAN  |PARTITIONED|
                                                -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                  -- EMPTY_TUPLE_SOURCE  |PARTITIONED|
                                    -- HASH_PARTITION_EXCHANGE [$$117]  |PARTITIONED|
                                      -- STREAM_PROJECT  |PARTITIONED|
                                        -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                          -- HYBRID_HASH_JOIN [$$100, $$99][$$112, $$117]  |PARTITIONED|
                                            -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                              -- STREAM_PROJECT  |PARTITIONED|
                                                -- ASSIGN  |PARTITIONED|
                                                  -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                    -- DATASOURCE_SCAN  |PARTITIONED|
                                                      -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                        -- EMPTY_TUPLE_SOURCE  |PARTITIONED|
                                            -- HASH_PARTITION_EXCHANGE [$$117, $$112]  |PARTITIONED|
                                              -- STREAM_PROJECT  |PARTITIONED|
                                                -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                  -- HYBRID_HASH_JOIN [$$101][$$112]  |PARTITIONED|
                                                    -- HASH_PARTITION_EXCHANGE [$$101]  |PARTITIONED|
                                                      -- STREAM_PROJECT  |PARTITIONED|
                                                        -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                          -- HYBRID_HASH_JOIN [$$108][$$102]  |PARTITIONED|
                                                            -- HASH_PARTITION_EXCHANGE [$$108]  |PARTITIONED|
                                                              -- STREAM_PROJECT  |PARTITIONED|
                                                                -- ASSIGN  |PARTITIONED|
                                                                  -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                                    -- DATASOURCE_SCAN  |PARTITIONED|
                                                                      -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                                        -- EMPTY_TUPLE_SOURCE  |PARTITIONED|
                                                            -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                              -- STREAM_PROJECT  |PARTITIONED|
                                                                -- ASSIGN  |PARTITIONED|
                                                                  -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                                    -- DATASOURCE_SCAN  |PARTITIONED|
                                                                      -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                                        -- EMPTY_TUPLE_SOURCE  |PARTITIONED|
                                                    -- HASH_PARTITION_EXCHANGE [$$112]  |PARTITIONED|
                                                      -- STREAM_PROJECT  |PARTITIONED|
                                                        -- ASSIGN  |PARTITIONED|
                                                          -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                            -- DATASOURCE_SCAN  |PARTITIONED|
                                                              -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                                -- EMPTY_TUPLE_SOURCE  |PARTITIONED|

Regards
Khurram

Original issue reported on code.google.com by khfaraaz82 on 19 Mar 2012 at 9:03

GoogleCodeExporter commented 8 years ago 
Hi Yingyi,

Another test tpch/q10_returned_item.aql fails with same problem as described 
above.

Here is a snippet from the failing query

...
let $revenue := sum(for $i in $locn return $i.l_extendedprice * (1 - 
$i.l_discount))
order by $revenue desc
limit 20
return {
        "c_custkey": $c_custkey, 
        "c_name": $c_name,
        "revenue": $revenue,
        "c_acctbal": $c_acctbal, 
        "n_name": $n_name, 
        "c_address": $c_address, 
        "c_phone": $c_phone, 
        "c_comment": $c_comment
}

The difference is in the precision of value returned by sum aggregate function
Look at the value for the revenue field to note the difference.

java.lang.AssertionError: Result for 
src/test/resources/runtimets/queries/tpch/q10_returned_item.aql changed at line 
15:
< { "c_custkey": 53, "c_name": "Customer#000000053", "revenue": 92568.9124d, 
"c_acctbal": 4113.64d, "n_name": "MOROCCO", "c_address": 
"HnaxHzTfFTZs8MuCpJyTbZ47Cm4wFOOgib", "c_phone": "25-168-852-5363", 
"c_comment": "ar accounts are. even foxes are blithely. fluffily pending 
deposits boost" }
> { "c_custkey": 53, "c_name": "Customer#000000053", "revenue": 
92568.91239999999d, "c_acctbal": 4113.64d, "n_name": "MOROCCO", "c_address": 
"HnaxHzTfFTZs8MuCpJyTbZ47Cm4wFOOgib", "c_phone": "25-168-852-5363", 
"c_comment": "ar accounts are. even foxes are blithely. fluffily pending 
deposits boost" }

Thanks
Khurram

Original comment by khfaraaz82 on 19 Mar 2012 at 9:12

GoogleCodeExporter commented 8 years ago 
Khurram,

Can you change the queries so that we round the result of the sum and 
then persist the new results.

Original comment by vinay...@gmail.com on 19 Mar 2012 at 9:24

GoogleCodeExporter commented 8 years ago 
fixed in r68

Original comment by buyingyi@gmail.com on 27 Mar 2012 at 5:18