ExternalGroupOperator failing with NPE

[GoogleCodeExporter]

Description of the desired feature(s) (incl. use cases, if applicable)

SEVERE: Test 
"src/test/resources/runtimets/queries/fuzzyjoin/dblp-2.1_5.3.1.aql" FAILED!
edu.uci.ics.hyracks.api.exceptions.HyracksException: Job Failed
    at edu.uci.ics.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:156)
    at edu.uci.ics.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:42)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:680)
Caused by: edu.uci.ics.hyracks.api.exceptions.HyracksException: Exception 
caught by thread: 
edu.uci.ics.hyracks.dataflow.std.group.ExternalGroupOperatorDescriptor$MergeActi
vity:TAID:TID:ANID:ODID:2:1:1:0:0
java.lang.NullPointerException
    at edu.uci.ics.hyracks.dataflow.std.group.ExternalGroupOperatorDescriptor$MergeActivity$1.initialize(ExternalGroupOperatorDescriptor.java:343)
    at edu.uci.ics.hyracks.control.nc.Task.run(Task.java:229)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:680)

Original issue reported on code.google.com by RamanGro...@gmail.com on 22 Feb 2012 at 2:51

Attachments:

• dblp-2_5.3.1.aql

[GoogleCodeExporter]

Raman - I think this may be misfiled - you've filed it as a Hyracks issue.  I 
will leave it up to Vinayak, but I would think all issues like this ont should 
start as ASTERIX issues, since this is an ASTERIX test - otherwise we cannot 
see them at that level.  If they turn out to have a Hyracks cause, a Hyracks 
issue can be filed - but only then - and only if there is a Hyracks-only 
reproducer for the issue, I would say, else keep it at the AQL/ADM level and 
track it w/ASTERIX. Make sense?

Original comment by dtab...@gmail.com on 22 Feb 2012 at 3:17

[GoogleCodeExporter]

PS - So please refile up above, unless Vinayak disagrees; thx!

Original comment by dtab...@gmail.com on 22 Feb 2012 at 3:18

[GoogleCodeExporter]

Raman,

Please file this issue as an Asterix issue as Mike suggests. Two reasons for 
that:
1. The visibility argument that Mike makes -- this issue is not reproducible at 
the Hyracks level
2. An NPE in a Hyracks operator does not mean that its a Hyracks bug -- the 
arguments passed in could have the bug.

However, having said all the above, in this specific case it turns out the bug 
is indeed in Hyracks -- specifically in the 
SerializableAggregatorDescriptorFactory class used by the count aggregator.

The createAggregateStates() call in the Aggregator returns null.

Assigning the bug to Jarod.

@Jarod, My understanding is that the AggregatorState object is optionally 
created by an aggregator if it needs additional state, is that correct? If so, 
The EGOD must call close only if the state was non-null at line 343.

Original comment by vinay...@gmail.com on 22 Feb 2012 at 7:14

[GoogleCodeExporter]

Thanks for the prompt responses. 
My initial investigation into the hyracks code suggested it to be hyracks bug 
and hence logged with Hyracks. 

I have put it up on Asterix issue list (Issue no 36)
Please update the bug when the fix is committed so that I can sync up. 

Original comment by RamanGro...@gmail.com on 22 Feb 2012 at 7:21

[GoogleCodeExporter]

What's the version of the hyracks used by ASTERIX? In the dev_next version, the 
AggregateState should never be null (it is created and returned by the count 
aggregator, but when additional state is not used, it is just an integer state 
never being used).

Original comment by jarod...@gmail.com on 22 Feb 2012 at 8:18

• Changed state: Started

[GoogleCodeExporter]

This issue was closed by revision r1226.

Original comment by jarod...@gmail.com on 22 Feb 2012 at 8:42

• Changed state: Fixed

[GoogleCodeExporter]

The fix has been committed.

Original comment by jarod...@gmail.com on 22 Feb 2012 at 8:43