consecutive index join fails with NPE

[GoogleCodeExporter]

The primary key of dataset TwitterUsers is screen-name,
the primary key of dataset TwitterMessages is tweetid.

The record type of TwitterUsers:
create type TwitterUserType as {
screen-name: string,
lang: string,
friends_count: int32,
statuses_count: int32,
name: string,
followers_count: int32
}

The record type of TwitterMessages:
create type TweetMessageType as {
tweetid: int64,     
user: string,
sender-location: point,
send-time: datetime,
forward-from: int64,
retweet-from: int64,
referred-topics: {{ string }},
message-text: string
}

Query:

for $tu in dataset TwitterUsers
for $tm in dataset TwitterMessages
for $r in dataset results
    where 
       $r.vertexid /*+ indexnl */ = $tm.tweetid 
       and
       $tm.user  /*+ indexnl */ = $tu.screen-name             
return 1

Stacktrace:
SEVERE: null
java.lang.NullPointerException
    at edu.uci.ics.asterix.metadata.entities.Index.getNonNullableType(Index.java:133)
    at edu.uci.ics.asterix.optimizer.rules.am.AbstractIntroduceAccessMethodRule.isMatched(AbstractIntroduceAccessMethodRule.java:313)
    at edu.uci.ics.asterix.optimizer.rules.am.AbstractIntroduceAccessMethodRule.pruneIndexCandidates(AbstractIntroduceAccessMethodRule.java:259)
    at edu.uci.ics.asterix.optimizer.rules.am.AbstractIntroduceAccessMethodRule.pruneIndexCandidates(AbstractIntroduceAccessMethodRule.java:116)
    at edu.uci.ics.asterix.optimizer.rules.am.IntroduceJoinAccessMethodRule.rewritePost(IntroduceJoinAccessMethodRule.java:131)
    at edu.uci.ics.hyracks.algebricks.core.rewriter.base.AbstractRuleController.rewriteOperatorRef(AbstractRuleController.java:122)
    at edu.uci.ics.hyracks.algebricks.core.rewriter.base.AbstractRuleController.rewriteOperatorRef(AbstractRuleController.java:96)
    at edu.uci.ics.hyracks.algebricks.core.rewriter.base.AbstractRuleController.rewriteOperatorRef(AbstractRuleController.java:96)
    at edu.uci.ics.hyracks.algebricks.core.rewriter.base.AbstractRuleController.rewriteOperatorRef(AbstractRuleController.java:96)
    at edu.uci.ics.hyracks.algebricks.compiler.rewriter.rulecontrollers.SequentialFixpointRuleController.rewriteWithRuleCollection(SequentialFixpointRuleController.java:49)
    at edu.uci.ics.hyracks.algebricks.core.rewriter.base.HeuristicOptimizer.runOptimizationSets(HeuristicOptimizer.java:91)
    at edu.uci.ics.hyracks.algebricks.core.rewriter.base.HeuristicOptimizer.optimize(HeuristicOptimizer.java:78)
    at edu.uci.ics.hyracks.algebricks.compiler.api.HeuristicCompilerFactoryBuilder$1$1.optimize(HeuristicCompilerFactoryBuilder.java:83)
    at edu.uci.ics.asterix.api.common.APIFramework.compileQuery(APIFramework.java:275)
    at edu.uci.ics.asterix.aql.translator.AqlTranslator.rewriteCompileQuery(AqlTranslator.java:1891)
    at edu.uci.ics.asterix.aql.translator.AqlTranslator.handleQuery(AqlTranslator.java:2459)
    at edu.uci.ics.asterix.aql.translator.AqlTranslator.compileAndExecute(AqlTranslator.java:367)
    at edu.uci.ics.asterix.api.http.servlet.APIServlet.doPost(APIServlet.java:111)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:754)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:970)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:904)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
    at org.eclipse.jetty.server.Server.handle(Server.java:347)
    at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:439)
    at org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:924)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:781)
    at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220)
    at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:43)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:545)
    at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:43)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
    at java.lang.Thread.run(Thread.java:744)

Original issue reported on code.google.com by buyingyi@gmail.com on 5 Aug 2015 at 5:39

[GoogleCodeExporter]

Can you also show me the DDL for the dataset "results"?

Original comment by wangs...@gmail.com on 5 Aug 2015 at 5:32

[GoogleCodeExporter]

Here you go:

create type Result as {vertexid: int64,  rank: double};

create temporary dataset results(Result) primary key vertexid;

Original comment by buyingyi@gmail.com on 5 Aug 2015 at 5:55

[GoogleCodeExporter]

Once the first join condition is transformed as an index-nested loop join, 
there are two datasources (unnest-map and datasource-scan). However, in the 
current codebase, we assume there is only one datasource in the sub-tree.  I 
have added additional datasource for the subtree if there are two. I will 
verify the result with the sample data that Yingyi will provide.

Original comment by wangs...@gmail.com on 6 Aug 2015 at 8:22

• Changed state: FixedInBranch

[GoogleCodeExporter]

sample data attached.

Original comment by buyingyi@gmail.com on 8 Aug 2015 at 1:36

Attachments:

• results
• tw_messages
• tw_users

[GoogleCodeExporter]

Thanks. It works as expected. I am going to submit the new code to the 
code-review.

Original comment by wangs...@gmail.com on 8 Aug 2015 at 2:47

[GoogleCodeExporter]

Great! Thanks!!

Best,
Yingyi

Original comment by buyingyi@gmail.com on 8 Aug 2015 at 2:49