search

lwhay / asterixdb

Automatically exported from code.google.com/p/asterixdb
0 stars 0 forks source link

NullPointerException during feed ingestion into a dataset with a secondary index #711

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?

1. In master branch, add an index creation statement in the ddl file 
(asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/feeds_04/feeds
_04.1.ddl.aql) as follows:
---------------------------
/*
 * Description  : Create a feed dataset that uses the feed simulator adapter. 
                  The feed simulator simulates feed from a file in the HDFS. 
                  Begin ingestion and verify contents of the dataset post completion.  
 * Expected Res : Success
 * Date         : 24th Dec 2012
 */
drop dataverse feeds if exists;
create dataverse feeds;
use dataverse feeds;

create type TweetType as closed {
  id: string,
  username : string,
  location : string,
  text : string,
  timestamp : string
}      

create dataset Tweets(TweetType)
primary key id;

create index usernameIdx on Tweets(username) type btree;

create feed TweetFeed
using file_feed
(("fs"="hdfs"),("hdfs"="hdfs://127.0.0.1:31888"),("path"="/asterix/obamatweets.a
dm"),("format"="adm"),("input-format"="text-input-format"),("type-name"="TweetTy
pe"),("tuple-interval"="10"));
----------------------------------

2. run ExecutionTest

What is the expected output? What do you see instead?
It should pass but it shows the NullPointerException as follows.

ACTIVE FEEDS TweetFeed
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Did not find a saved state from a previous zombie, starting a new 
instance for STORAGE node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Did not find a saved state from a previous zombie, starting a new 
instance for STORAGE node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Did not find a saved state from a previous zombie, starting a new 
instance for COMMIT node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Did not find a saved state from a previous zombie, starting a new 
instance for COMMIT node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Did not find a saved state from a previous zombie, starting a new 
instance for STORAGE node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Did not find a saved state from a previous zombie, starting a new 
instance for STORAGE node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Did not find a saved state from a previous zombie, starting a new 
instance for COMMIT node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Did not find a saved state from a previous zombie, starting a new 
instance for COMMIT node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Did not find a saved state from a previous zombie, starting a new 
instance for COMPUTE node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Retreived state from the zombie instance from previous execution for 
COMPUTE node.
java.lang.NullPointerException
    at edu.uci.ics.asterix.metadata.feeds.FeedFrameWriter.reset(FeedFrameWriter.java:382)
    at edu.uci.ics.asterix.metadata.feeds.FeedIntakeOperatorNodePushable.initialize(FeedIntakeOperatorNodePushable.java:99)
    at edu.uci.ics.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:81)
    at edu.uci.ics.hyracks.control.nc.Task.run(Task.java:234)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Retreived state from the zombie instance from previous execution for 
STORAGE node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Retreived state from the zombie instance from previous execution for 
STORAGE node.
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
java.lang.NullPointerException
    at edu.uci.ics.asterix.metadata.feeds.FeedIntakeOperatorNodePushable.initialize(FeedIntakeOperatorNodePushable.java:138)
    at edu.uci.ics.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:81)
    at edu.uci.ics.hyracks.control.nc.Task.run(Task.java:234)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.NullPointerException
    at edu.uci.ics.asterix.metadata.feeds.FeedFrameWriter.reset(FeedFrameWriter.java:382)
    at edu.uci.ics.asterix.metadata.feeds.FeedIntakeOperatorNodePushable.initialize(FeedIntakeOperatorNodePushable.java:99)
    ... 5 more
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Retreived state from the zombie instance from previous execution for 
COMMIT node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Retreived state from the zombie instance from previous execution for 
COMMIT node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Did not find a saved state from a previous zombie, starting a new 
instance for STORAGE node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Retreived state from the zombie instance from previous execution for 
STORAGE node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Retreived state from the zombie instance from previous execution for 
COMMIT node.
Feb 05, 2014 6:01:35 PM 
edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushab
le open
WARNING: Retreived state from the zombie instance from previous execution for 
COMMIT node.
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
java.lang.InterruptedException
    at edu.uci.ics.hyracks.control.nc.Task.pushFrames(Task.java:332)
    at edu.uci.ics.hyracks.control.nc.Task.run(Task.java:269)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Caused by: edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
java.lang.InterruptedException
    at edu.uci.ics.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:107)
    at edu.uci.ics.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:38)
    at edu.uci.ics.hyracks.control.nc.Task.pushFrames(Task.java:311)
    ... 4 more
Caused by: java.lang.InterruptedException
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:503)
    at edu.uci.ics.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:105)
    ... 6 more
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
java.lang.InterruptedException
    at edu.uci.ics.hyracks.control.nc.Task.pushFrames(Task.java:332)
    at edu.uci.ics.hyracks.control.nc.Task.run(Task.java:269)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Caused by: edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
java.lang.InterruptedException
    at edu.uci.ics.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:107)
    at edu.uci.ics.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:38)
    at edu.uci.ics.hyracks.control.nc.Task.pushFrames(Task.java:311)
    ... 4 more
Caused by: java.lang.InterruptedException
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:503)
    at edu.uci.ics.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:105)
    ... 6 more
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
java.lang.InterruptedException
    at edu.uci.ics.hyracks.control.nc.Task.pushFrames(Task.java:332)
    at edu.uci.ics.hyracks.control.nc.Task.run(Task.java:269)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Caused by: edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
java.lang.InterruptedException
    at edu.uci.ics.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:107)
    at edu.uci.ics.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:38)
    at edu.uci.ics.hyracks.control.nc.Task.pushFrames(Task.java:311)
    ... 4 more
Caused by: java.lang.InterruptedException
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:503)
    at edu.uci.ics.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:105)
    ... 6 more
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
java.lang.InterruptedException
    at edu.uci.ics.hyracks.control.nc.Task.pushFrames(Task.java:332)
    at edu.uci.ics.hyracks.control.nc.Task.run(Task.java:269)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Caused by: edu.uci.ics.hyracks.api.exceptions.HyracksDataException: 
java.lang.InterruptedException
    at edu.uci.ics.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:107)
    at edu.uci.ics.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:38)
    at edu.uci.ics.hyracks.control.nc.Task.pushFrames(Task.java:311)
    ... 4 more
Caused by: java.lang.InterruptedException
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:503)
    at edu.uci.ics.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:105)
    ... 6 more
Feb 05, 2014 6:01:35 PM edu.uci.ics.hyracks.control.cc.work.JobCleanupWork run
WARNING: Ignoring duplicate cleanup for JobRun with id: JID:9
Feb 05, 2014 6:01:35 PM edu.uci.ics.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 688128 bytes
Feb 05, 2014 6:01:35 PM edu.uci.ics.hyracks.control.cc.work.JobCleanupWork run
WARNING: Ignoring duplicate cleanup for JobRun with id: JID:9
Feb 05, 2014 6:01:35 PM edu.uci.ics.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 524288 bytes
Feb 05, 2014 6:01:35 PM edu.uci.ics.hyracks.control.cc.work.JobCleanupWork run
WARNING: Ignoring duplicate cleanup for JobRun with id: JID:9
Feb 05, 2014 6:01:35 PM edu.uci.ics.hyracks.control.cc.work.JobCleanupWork run
WARNING: Ignoring duplicate cleanup for JobRun with id: JID:9
java.lang.NullPointerException
    at edu.uci.ics.asterix.metadata.feeds.FeedMetaOperatorDescriptor$FeedMetaNodePushable.nextFrame(FeedMetaOperatorDescriptor.java:187)
    at edu.uci.ics.asterix.metadata.feeds.FeedFrameWriter.nextFrame(FeedFrameWriter.java:155)
    at edu.uci.ics.hyracks.dataflow.common.comm.util.FrameUtils.flushFrame(FrameUtils.java:38)
    at edu.uci.ics.asterix.tools.external.data.RateControlledTupleParser.parse(RateControlledFileSystemBasedAdapterFactory.java:263)
    at edu.uci.ics.asterix.external.dataset.adapter.FileSystemBasedAdapter.start(FileSystemBasedAdapter.java:50)
    at edu.uci.ics.asterix.metadata.feeds.AdapterRuntimeManager$AdapterExecutor.run(AdapterRuntimeManager.java:138)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Feb 05, 2014 6:01:35 PM edu.uci.ics.asterix.metadata.feeds.FeedFrameWriter 
nextFrame
SEVERE: Unable to write frame  on behalf of 
edu.uci.ics.asterix.metadata.feeds.FeedIntakeOperatorNodePushable@159273aa:
java.lang.NullPointerException

Original issue reported on code.google.com by kiss...@gmail.com on 6 Feb 2014 at 2:03

GoogleCodeExporter commented 9 years ago 
fixed in branch: raman/master_feeds_711

pending review by Sattam.

Original comment by RamanGro...@gmail.com on 6 Feb 2014 at 6:47

GoogleCodeExporter commented 9 years ago 
Could you also check whether Rtree and InvertedIndex have the same issue or not?
It will be good to add test cases having Btree, Rtree, and InvertedIndex as a 
secondary index.

Original comment by kiss...@gmail.com on 6 Feb 2014 at 5:07

GoogleCodeExporter commented 9 years ago 
Added additional test cases and merged to master

Original comment by RamanGro...@gmail.com on 7 Feb 2014 at 7:01