search

ilovesoup / hyracks

Automatically exported from code.google.com/p/hyracks
Apache License 2.0
0 stars 0 forks source link

JobActivityGraph was null #69

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
I got this exception when I run some Hivesterix queries:

java.lang.NullPointerException: JobActivityGraph was null
       at edu.uci.ics.hyracks.control.nc.work.StartTasksWork.getOrCreateLocalJoblet(StartTasksWork.java:168)
       at edu.uci.ics.hyracks.control.nc.work.StartTasksWork.doRun(StartTasksWork.java:89)
       at edu.uci.ics.hyracks.control.common.work.SynchronizableWork.run(SynchronizableWork.java:32)
       at edu.uci.ics.hyracks.control.common.work.WorkQueue$WorkerThread.run(WorkQueue.java:115)
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing:
edu.uci.ics.hyracks.control.nc.work.NotifyTaskFailureWork@30b95f2
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing:
edu.uci.ics.hyracks.control.nc.work.NotifyTaskFailureWork@6da1446d
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing:
edu.uci.ics.hyracks.control.nc.work.NotifyTaskFailureWork@51037959
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue schedule
INFO: Scheduling:
edu.uci.ics.hyracks.control.nc.work.NotifyTaskFailureWork@2533b5db
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing:
edu.uci.ics.hyracks.control.nc.work.NotifyTaskFailureWork@2533b5db
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue schedule
INFO: Scheduling:
edu.uci.ics.hyracks.control.nc.work.NotifyTaskFailureWork@69fed1f3
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing:
edu.uci.ics.hyracks.control.nc.work.NotifyTaskFailureWork@69fed1f3
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue schedule
INFO: Scheduling: edu.uci.ics.hyracks.control.nc.work.AbortTasksWork@6f76dd71
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing: edu.uci.ics.hyracks.control.nc.work.AbortTasksWork@6f76dd71
Jun 3, 2012 3:52:33 AM edu.uci.ics.hyracks.control.nc.work.AbortTasksWork doRun
INFO: Aborting Tasks: JID:0:[TAID:TID:ANID:ODID:7:0:9:1,
TAID:TID:ANID:ODID:6:1:9:1]
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue schedule
INFO: Scheduling:
edu.uci.ics.hyracks.control.nc.work.NotifyTaskFailureWork@64c45ee2
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing:
edu.uci.ics.hyracks.control.nc.work.NotifyTaskFailureWork@64c45ee2
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue schedule
INFO: Scheduling: edu.uci.ics.hyracks.control.nc.work.StartTasksWork@693f87b1
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing: edu.uci.ics.hyracks.control.nc.work.StartTasksWork@693f87b1
java.lang.NullPointerException: JobActivityGraph was null
       at edu.uci.ics.hyracks.control.nc.work.StartTasksWork.getOrCreateLocalJoblet(StartTasksWork.java:168)
       at edu.uci.ics.hyracks.control.nc.work.StartTasksWork.doRun(StartTasksWork.java:89)
       at edu.uci.ics.hyracks.control.common.work.SynchronizableWork.run(SynchronizableWork.java:32)
       at edu.uci.ics.hyracks.control.common.work.WorkQueue$WorkerThread.run(WorkQueue.java:115)
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue schedule
INFO: Scheduling: edu.uci.ics.hyracks.control.nc.work.StartTasksWork@49e5ab72
Jun 3, 2012 3:52:33 AM
edu.uci.ics.hyracks.control.common.work.WorkQueue$WorkerThread run
INFO: Executing: edu.uci.ics.hyracks.control.nc.work.StartTasksWork@49e5ab72

An example query plan is:
write [%0->$$34, %0->$$37, %0->$$35, %0->$$36]
-- SINK_WRITE  |UNPARTITIONED|
  project ([$$34, $$37, $$35, $$36])
  -- STREAM_PROJECT  |PARTITIONED|
    limit 10
    -- STREAM_LIMIT  |UNPARTITIONED|
      limit 10
      -- STREAM_LIMIT  |UNPARTITIONED|
        exchange
        -- SORT_MERGE_EXCHANGE [$$37(DESC), $$35(ASC) ]  |PARTITIONED|
          limit 10
          -- STREAM_LIMIT  |LOCAL|
            exchange
            -- ONE_TO_ONE_EXCHANGE  |LOCAL|
              order (DESC, %0->$$37) (ASC, %0->$$35)
              -- STABLE_SORT [$$37(DESC), $$35(ASC)]  |LOCAL|
                exchange
                -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                  group by ([$$34 := %0->$$40; $$35 := %0->$$41; $$36 := %0->$$42]) decor ([]) {
                            aggregate [$$37] <- [function-call: hive:sum(FINAL), Args:[%0->$$39]]
                            -- AGGREGATE  |LOCAL|
                              nested tuple source
                              -- NESTED_TUPLE_SOURCE  |LOCAL|
                         }
                  -- EXTERNAL_GROUP_BY[$$40, $$41, $$42]  |PARTITIONED|
                    exchange
                    -- HASH_PARTITION_EXCHANGE [$$40, $$41, $$42]  |PARTITIONED|
                      group by ([$$40 := %0->$$1; $$41 := %0->$$29; $$42 := %0->$$32]) decor ([]) {
                                aggregate [$$39] <- [function-call: hive:sum(PARTIAL1), Args:[function-call: hive:org.apache.hadoop.hive.ql.udf.UDFOPMultiply, Args:[%0->$$6, function-call: hive:org.apache.hadoop.hive.ql.udf.UDFOPMinus, Args:[1, %0->$$7]]]]
                                -- AGGREGATE  |LOCAL|
                                  nested tuple source
                                  -- NESTED_TUPLE_SOURCE  |LOCAL|
                             }
                      -- EXTERNAL_GROUP_BY[$$1, $$29, $$32]  |LOCAL|
                        exchange
                        -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                          project ([$$1, $$29, $$32, $$6, $$7])
                          -- STREAM_PROJECT  |PARTITIONED|
                            exchange
                            -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                              join (function-call: algebricks:eq, Args:[%0->$$1, %0->$$25])
                              -- HYBRID_HASH_JOIN [$$1][$$25]  |PARTITIONED|
                                exchange
                                -- HASH_PARTITION_EXCHANGE [$$1]  |PARTITIONED|
                                  project ([$$1, $$6, $$7])
                                  -- STREAM_PROJECT  |PARTITIONED|
                                    select (function-call: algebricks:and, Args:[function-call: algebricks:gt, Args:[%0->$$11, 1995-03-15], function-call: algebricks:gt, Args:[%0->$$11, 1995-03-15]])
                                    -- STREAM_SELECT  |PARTITIONED|
                                      exchange
                                      -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                        data-scan [$$1, $$6, $$7, $$11]<-[$$1, $$2, $$3, $$4, $$5, $$6, $$7, $$8, $$9, $$10, $$11, $$12, $$13, $$14, $$15, $$16] <- default.lineitem
                                        -- DATASOURCE_SCAN  |PARTITIONED|
                                          exchange
                                          -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                            empty-tuple-source
                                            -- EMPTY_TUPLE_SOURCE  |PARTITIONED|
                                exchange
                                -- HASH_PARTITION_EXCHANGE [$$25]  |PARTITIONED|
                                  project ([$$25, $$29, $$32])
                                  -- STREAM_PROJECT  |PARTITIONED|
                                    exchange
                                    -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                      join (function-call: algebricks:eq, Args:[%0->$$26, %0->$$17])
                                      -- HYBRID_HASH_JOIN [$$26][$$17]  |PARTITIONED|
                                        exchange
                                        -- HASH_PARTITION_EXCHANGE [$$26]  |PARTITIONED|
                                          select (function-call: algebricks:and, Args:[function-call: algebricks:lt, Args:[%0->$$29, 1995-03-15], function-call: algebricks:lt, Args:[%0->$$29, 1995-03-15]])
                                          -- STREAM_SELECT  |PARTITIONED|
                                            exchange
                                            -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                              data-scan [$$32, $$25, $$26, $$29]<-[$$25, $$26, $$27, $$28, $$29, $$30, $$31, $$32, $$33] <- default.orders
                                              -- DATASOURCE_SCAN  |PARTITIONED|
                                                exchange
                                                -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                  empty-tuple-source
                                                  -- EMPTY_TUPLE_SOURCE  |PARTITIONED|
                                        exchange
                                        -- HASH_PARTITION_EXCHANGE [$$17]  |PARTITIONED|
                                          project ([$$17])
                                          -- STREAM_PROJECT  |PARTITIONED|
                                            select (function-call: algebricks:eq, Args:[%0->$$23, BUILDING])
                                            -- STREAM_SELECT  |PARTITIONED|
                                              exchange
                                              -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                data-scan [$$17, $$23]<-[$$17, $$18, $$19, $$20, $$21, $$22, $$23, $$24] <- default.customer
                                                -- DATASOURCE_SCAN  |PARTITIONED|
                                                  exchange
                                                  -- ONE_TO_ONE_EXCHANGE  |PARTITIONED|
                                                    empty-tuple-source
                                                    -- EMPTY_TUPLE_SOURCE  |PARTITIONED|

Original issue reported on code.google.com by buyingyi@gmail.com on 3 Jun 2012 at 11:13

GoogleCodeExporter commented 9 years ago

Original comment by buyingyi@gmail.com on 3 Jun 2012 at 1:03

GoogleCodeExporter commented 9 years ago

Original comment by buyingyi@gmail.com on 3 Jun 2012 at 9:28

GoogleCodeExporter commented 9 years ago 
The problem is only in failure-triggered job restart. It does not show up in a 
normal job execution.

Original comment by buyingyi@gmail.com on 6 Jun 2012 at 9:36