search

lwhay / asterixdb

Automatically exported from code.google.com/p/asterixdb
0 stars 0 forks source link

group by with a nullable field returns an IndexOutOfBoundsException #756

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
If your type contains a field which is nullable, and you group by this field, 
then you will get an IndexOutOfBoundsException exception.
Note that the data doesn't have to contain a null field, the type just needs to 
be nullable to reproduce the error.

Reproduction:
=============
Change the FacebokUserType to have a nullable field (below I chose name):
(note that even if the 
create type FacebookUserType as closed {
        id: int32,
        alias: string,
        name: string?,
        user-since: datetime,
        friend-ids: {{ int32 }},
        employment: [EmploymentType]
}

for $user in dataset FacebookUsers
let $t := $user.name
group by $group_key:=$user.id with $t
return {
"id":$group_key,
"name":$t
}

Stack:
=======
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
edu.uci.ics.hyracks.algebricks.common.exceptions.AlgebricksException: 
java.lang.IllegalStateException: java.lang.IllegalStateException: 
java.lang.IndexOutOfBoundsException

    at edu.uci.ics.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:207)
    at edu.uci.ics.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:44)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Caused by: edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
edu.uci.ics.hyracks.algebricks.common.exceptions.AlgebricksException: 
java.lang.IllegalStateException: java.lang.IllegalStateException: 
java.lang.IndexOutOfBoundsException
    at edu.uci.ics.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:41)
    at edu.uci.ics.hyracks.control.nc.Task.run(Task.java:291)
    ... 3 more
Caused by: edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
edu.uci.ics.hyracks.algebricks.common.exceptions.AlgebricksException: 
java.lang.IllegalStateException: java.lang.IllegalStateException: 
java.lang.IndexOutOfBoundsException
    at edu.uci.ics.hyracks.algebricks.runtime.serializer.ResultSerializerFactoryProvider$1$1.appendTuple(ResultSerializerFactoryProvider.java:69)
    at edu.uci.ics.hyracks.dataflow.std.result.ResultWriterOperatorDescriptor$1.nextFrame(ResultWriterOperatorDescriptor.java:95)
    at edu.uci.ics.hyracks.dataflow.common.comm.util.FrameUtils.flushFrame(FrameUtils.java:38)
    at edu.uci.ics.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendProjectionToFrame(AbstractOneInputOneOutputOneFramePushRuntime.java:71)
    at edu.uci.ics.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendProjectionToFrame(AbstractOneInputOneOutputOneFramePushRuntime.java:65)
    at edu.uci.ics.hyracks.algebricks.runtime.operators.std.StreamProjectRuntimeFactory$1.nextFrame(StreamProjectRuntimeFactory.java:70)
    at edu.uci.ics.hyracks.dataflow.common.comm.util.FrameUtils.flushFrame(FrameUtils.java:38)
    at edu.uci.ics.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.close(AbstractOneInputOneOutputOneFramePushRuntime.java:38)
    at edu.uci.ics.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.close(AbstractOneInputOneOutputOneFramePushRuntime.java:41)
    at edu.uci.ics.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.close(AlgebricksMetaOperatorDescriptor.java:141)
    at edu.uci.ics.hyracks.dataflow.std.group.preclustered.PreclusteredGroupWriter.close(PreclusteredGroupWriter.java:168)
    at edu.uci.ics.hyracks.dataflow.std.group.preclustered.PreclusteredGroupOperatorNodePushable.close(PreclusteredGroupOperatorNodePushable.java:82)
    at edu.uci.ics.hyracks.dataflow.std.sort.ExternalSortRunMerger.process(ExternalSortRunMerger.java:164)
    at edu.uci.ics.hyracks.dataflow.std.sort.ExternalSortOperatorDescriptor$MergeActivity$1.initialize(ExternalSortOperatorDescriptor.java:189)
    at edu.uci.ics.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:81)
    at edu.uci.ics.hyracks.control.nc.Task.run(Task.java:234)
    ... 3 more
Caused by: 
edu.uci.ics.hyracks.algebricks.common.exceptions.AlgebricksException: 
java.lang.IllegalStateException: java.lang.IllegalStateException: 
java.lang.IndexOutOfBoundsException
    at edu.uci.ics.asterix.dataflow.data.nontagged.printers.ARecordPrinterFactory$1.print(ARecordPrinterFactory.java:64)
    at edu.uci.ics.hyracks.algebricks.runtime.writers.PrinterBasedWriterFactory$1.printTuple(PrinterBasedWriterFactory.java:62)
    at edu.uci.ics.hyracks.algebricks.runtime.serializer.ResultSerializerFactoryProvider$1$1.appendTuple(ResultSerializerFactoryProvider.java:65)
    ... 18 more
Caused by: java.lang.IllegalStateException: java.lang.IllegalStateException: 
java.lang.IndexOutOfBoundsException
    at edu.uci.ics.asterix.om.pointables.ARecordPointable.set(ARecordPointable.java:263)
    at edu.uci.ics.asterix.dataflow.data.nontagged.printers.ARecordPrinterFactory$1.print(ARecordPrinterFactory.java:60)
    ... 20 more
Caused by: java.lang.IllegalStateException: java.lang.IndexOutOfBoundsException
    at edu.uci.ics.asterix.om.pointables.AListPointable.set(AListPointable.java:160)
    at edu.uci.ics.asterix.om.pointables.ARecordPointable.set(ARecordPointable.java:225)
    ... 21 more
Caused by: java.lang.IndexOutOfBoundsException
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:138)
    at edu.uci.ics.hyracks.data.std.util.ByteArrayAccessibleOutputStream.write(ByteArrayAccessibleOutputStream.java:48)
    at java.io.DataOutputStream.write(DataOutputStream.java:107)
    at edu.uci.ics.asterix.om.pointables.AListPointable.set(AListPointable.java:133)
    ... 22 more
Apr 07, 2014 3:00:39 PM edu.uci.ics.asterix.api.http.servlet.APIServlet doPost
SEVERE: Job failed on account of:
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
edu.uci.ics.hyracks.algebricks.common.exceptions.AlgebricksException: 
java.lang.IllegalStateException: java.lang.IllegalStateException: 
java.lang.IndexOutOfBoundsException

edu.uci.ics.hyracks.api.exceptions.HyracksException: Job failed on account of:
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
edu.uci.ics.hyracks.algebricks.common.exceptions.AlgebricksException: 
java.lang.IllegalStateException: java.lang.IllegalStateException: 
java.lang.IndexOutOfBoundsException

    at edu.uci.ics.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:207)
    at edu.uci.ics.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:44)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Caused by: edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
edu.uci.ics.hyracks.algebricks.common.exceptions.AlgebricksException: 
java.lang.IllegalStateException: java.lang.IllegalStateException: 
java.lang.IndexOutOfBoundsException
    at edu.uci.ics.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:41)
    at edu.uci.ics.hyracks.control.nc.Task.run(Task.java:291)
    ... 3 more
Caused by: edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
edu.uci.ics.hyracks.algebricks.common.exceptions.AlgebricksException: 
java.lang.IllegalStateException: java.lang.IllegalStateException: 
java.lang.IndexOutOfBoundsException
    at edu.uci.ics.hyracks.algebricks.runtime.serializer.ResultSerializerFactoryProvider$1$1.appendTuple(ResultSerializerFactoryProvider.java:69)
    at edu.uci.ics.hyracks.dataflow.std.result.ResultWriterOperatorDescriptor$1.nextFrame(ResultWriterOperatorDescriptor.java:95)
    at edu.uci.ics.hyracks.dataflow.common.comm.util.FrameUtils.flushFrame(FrameUtils.java:38)
    at edu.uci.ics.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendProjectionToFrame(AbstractOneInputOneOutputOneFramePushRuntime.java:71)
    at edu.uci.ics.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendProjectionToFrame(AbstractOneInputOneOutputOneFramePushRuntime.java:65)
    at edu.uci.ics.hyracks.algebricks.runtime.operators.std.StreamProjectRuntimeFactory$1.nextFrame(StreamProjectRuntimeFactory.java:70)
    at edu.uci.ics.hyracks.dataflow.common.comm.util.FrameUtils.flushFrame(FrameUtils.java:38)
    at edu.uci.ics.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.close(AbstractOneInputOneOutputOneFramePushRuntime.java:38)
    at edu.uci.ics.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.close(AbstractOneInputOneOutputOneFramePushRuntime.java:41)
    at edu.uci.ics.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.close(AlgebricksMetaOperatorDescriptor.java:141)
    at edu.uci.ics.hyracks.dataflow.std.group.preclustered.PreclusteredGroupWriter.close(PreclusteredGroupWriter.java:168)
    at edu.uci.ics.hyracks.dataflow.std.group.preclustered.PreclusteredGroupOperatorNodePushable.close(PreclusteredGroupOperatorNodePushable.java:82)
    at edu.uci.ics.hyracks.dataflow.std.sort.ExternalSortRunMerger.process(ExternalSortRunMerger.java:164)
    at edu.uci.ics.hyracks.dataflow.std.sort.ExternalSortOperatorDescriptor$MergeActivity$1.initialize(ExternalSortOperatorDescriptor.java:189)
    at edu.uci.ics.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:81)
    at edu.uci.ics.hyracks.control.nc.Task.run(Task.java:234)
    ... 3 more
Caused by: 
edu.uci.ics.hyracks.algebricks.common.exceptions.AlgebricksException: 
java.lang.IllegalStateException: java.lang.IllegalStateException: 
java.lang.IndexOutOfBoundsException
    at edu.uci.ics.asterix.dataflow.data.nontagged.printers.ARecordPrinterFactory$1.print(ARecordPrinterFactory.java:64)
    at edu.uci.ics.hyracks.algebricks.runtime.writers.PrinterBasedWriterFactory$1.printTuple(PrinterBasedWriterFactory.java:62)
    at edu.uci.ics.hyracks.algebricks.runtime.serializer.ResultSerializerFactoryProvider$1$1.appendTuple(ResultSerializerFactoryProvider.java:65)
    ... 18 more
Caused by: java.lang.IllegalStateException: java.lang.IllegalStateException: 
java.lang.IndexOutOfBoundsException
    at edu.uci.ics.asterix.om.pointables.ARecordPointable.set(ARecordPointable.java:263)
    at edu.uci.ics.asterix.dataflow.data.nontagged.printers.ARecordPrinterFactory$1.print(ARecordPrinterFactory.java:60)
    ... 20 more
Caused by: java.lang.IllegalStateException: java.lang.IndexOutOfBoundsException
    at edu.uci.ics.asterix.om.pointables.AListPointable.set(AListPointable.java:160)
    at edu.uci.ics.asterix.om.pointables.ARecordPointable.set(ARecordPointable.java:225)
    ... 21 more
Caused by: java.lang.IndexOutOfBoundsException
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:138)
    at edu.uci.ics.hyracks.data.std.util.ByteArrayAccessibleOutputStream.write(ByteArrayAccessibleOutputStream.java:48)
    at java.io.DataOutputStream.write(DataOutputStream.java:107)
    at edu.uci.ics.asterix.om.pointables.AListPointable.set(AListPointable.java:133)
    ... 22 more

Optimized plan:
================

distribute result [%0->$$12]
-- DISTRIBUTE_RESULT  |PARTITIONED|
  exchange 
  -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
    project ([$$12])
    -- STREAM_PROJECT  |PARTITIONED|
      assign [$$12] <- [function-call: asterix:closed-record-constructor, Args:[AString: {ts}, %0->$$10, AString: {u}, %0->$$11]]
      -- ASSIGN  |PARTITIONED|
        project ([$$10, $$11])
        -- STREAM_PROJECT  |PARTITIONED|
          exchange 
          -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
            group by ([$$3 := %0->$$5]) decor ([]) {
                      aggregate [$$10] <- [function-call: asterix:listify, Args:[%0->$$4]]
                      -- AGGREGATE  |LOCAL|
                        nested tuple source
                        -- NESTED_TUPLE_SOURCE  |LOCAL|
                   }
                   {
                      aggregate [$$11] <- [function-call: asterix:listify, Args:[%0->$$5]]
                      -- AGGREGATE  |LOCAL|
                        nested tuple source
                        -- NESTED_TUPLE_SOURCE  |LOCAL|
                   }
            -- PRE_CLUSTERED_GROUP_BY[$$5]  |PARTITIONED|
              exchange 
              -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                order (ASC, %0->$$5) 
                -- STABLE_SORT [$$5(ASC)]  |PARTITIONED|
                  exchange 
                  -- HASH_PARTITION_EXCHANGE [$$5]  |PARTITIONED|
                    project ([$$4, $$5])
                    -- STREAM_PROJECT  |PARTITIONED|
                      assign [$$5, $$4] <- [function-call: asterix:field-access-by-index, Args:[%0->$$0, AInt32: {1}], function-call: asterix:field-access-by-index, Args:[%0->$$0, AInt32: {6}]]
                      -- ASSIGN  |PARTITIONED|
                        project ([$$0])
                        -- STREAM_PROJECT  |PARTITIONED|
                          exchange 
                          -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                            data-scan []<-[$$14, $$0] <- kereno:page_views
                            -- DATASOURCE_SCAN  |PARTITIONED|
                              exchange 
                              -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                empty-tuple-source
                                -- EMPTY_TUPLE_SOURCE  |PARTITIONED|

Original issue reported on code.google.com by ker...@gmail.com on 8 Apr 2014 at 12:30

GoogleCodeExporter commented 9 years ago

Original comment by ildar.absalyamov on 25 Oct 2014 at 9:38

GoogleCodeExporter commented 9 years ago

Original comment by ildar.absalyamov on 6 Nov 2014 at 6:36