search

wala / WALA

T.J. Watson Libraries for Analysis, with frontends for Java, Android, and JavaScript, and may common static program analyses
http://github.com/wala/WALA
Eclipse Public License 2.0
758 stars 221 forks source link

Implicit parameter error involving Collector #144

Closed khatchad closed 7 years ago

khatchad commented 7 years ago

I am receiving an error when building a call graph of the following program:

package p;

import java.util.HashSet;
import java.util.Set;
import java.util.stream.Collectors;
import java.util.stream.Stream;

class A {
    void m() {
        Set<Object> set = new HashSet<>();
        Stream<Object> stream = set.parallelStream();
        stream.sorted(); 
        stream.collect(Collectors.toList()); // this call forces the error.
    }
}

I am at bb0f38338e00a9ffb37b711a08db10ea8a4dc332 and using JDK version 1.8.0_121.

Error details:

------------------ ERROR:
LocalPointerKey: [Node: synthetic < Primordial, Lwala/lambda$java$util$stream$Nodes$CollectorTask$OfRef$1, apply(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; > Context: CallStringContext: [ java.util.stream.Nodes$CollectorTask.onCompletion(Ljava/util/concurrent/CountedCompleter;)V@32 java.util.concurrent.CountedCompleter.tryComplete()V@15 ], v2]
Constant? false
   -- IR:
synthetic < Primordial, Lwala/lambda$java$util$stream$Nodes$CollectorTask$OfRef$1, apply(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; >
CFG:
BB0[0..0]
    -> BB1
    -> BB3
BB1[1..1]
    -> BB2
    -> BB3
BB2[2..2]
    -> BB3
BB3[-1..-2]
Instructions:
BB0
0   v2 = new <Application,Ljava/util/stream/Nodes$ConcNode>@1
BB1
1   invokespecial < Application, Ljava/util/stream/Nodes$ConcNode, <init>(Ljava/util/stream/Node;Ljava/util/stream/Node;)V > v2,v3,v4 @2 exception:v3
BB2
2   return v2                                
BB3

!ENTRY org.eclipse.core.jobs 4 2 2017-03-13 17:04:29.135
!MESSAGE An internal error occurred during: "...".
!STACK 0
com.ibm.wala.util.debug.UnimplementedError: How can parameter be implicit?
    at com.ibm.wala.util.debug.Assertions.UNREACHABLE(Assertions.java:55)
    at com.ibm.wala.ipa.callgraph.propagation.PropagationSystem.recordImplicitPointsToSet(PropagationSystem.java:246)
    at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$ConstraintVisitor.visitNew(SSAPropagationCallGraphBuilder.java:1222)
    at com.ibm.wala.ssa.SSANewInstruction.visit(SSANewInstruction.java:101)
    at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.addBlockInstructionConstraints(SSAPropagationCallGraphBuilder.java:292)
    at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.addNodeInstructionConstraints(SSAPropagationCallGraphBuilder.java:263)
    at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.unconditionallyAddConstraintsFromNode(SSAPropagationCallGraphBuilder.java:235)
    at com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.addConstraintsFromNode(SSAPropagationCallGraphBuilder.java:201)
    at com.ibm.wala.ipa.callgraph.propagation.PropagationCallGraphBuilder.addConstraintsFromNewNodes(PropagationCallGraphBuilder.java:338)
    at com.ibm.wala.ipa.callgraph.propagation.StandardSolver.solve(StandardSolver.java:58)
    at com.ibm.wala.ipa.callgraph.propagation.PropagationCallGraphBuilder.makeCallGraph(PropagationCallGraphBuilder.java:271)
    at com.ibm.wala.client.AbstractAnalysisEngine.buildCallGraph(AbstractAnalysisEngine.java:142)
...
juliandolby commented 7 years ago

Thanks for finding this bug. I was able to reproduce this crash on my system, and I have pushed a fix that seems to work for me. Let me know if it works for you too.

khatchad commented 7 years ago

No problem. With only sorted() it seems to work fine, so no regression there.

khatchad commented 7 years ago

However, when I add the collect() operation, I am in a seemingly endless loop. I'm investigating it further.

khatchad commented 7 years ago

Looks like the fix is in https://github.com/wala/WALA/commit/7e3066b6cda07ea6e977495b82cb0d2a202bb0c2.

juliandolby commented 7 years ago

does that mean that the code in issue 144 works for you?

On Mar 14, 2017, at 5:45 PM, Raffi Khatchadourian notifications@github.com wrote:

Looks like the fix is in 7e3066b https://github.com/wala/WALA/commit/7e3066b6cda07ea6e977495b82cb0d2a202bb0c2.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/wala/WALA/issues/144#issuecomment-286570805, or mute the thread https://github.com/notifications/unsubscribe-auth/ABk3fnF_7Rxbtrltv5EKHcmafl_RO-Zaks5rlwpkgaJpZM4Mb45K.

khatchad commented 7 years ago

I don't think so because I'm in a loop with the fix. I'm trying to localize where it's happening.

khatchad commented 7 years ago

Looks like I'm stuck in the call graph construction ...

juliandolby commented 7 years ago

how are you running this? it builds a call graph fine for me, based on the code you sent.

On Mar 14, 2017, at 5:52 PM, Raffi Khatchadourian notifications@github.com wrote:

Looks like I'm stuck in the call graph construction ...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/wala/WALA/issues/144#issuecomment-286572714, or mute the thread https://github.com/notifications/unsubscribe-auth/ABk3fsii49wuKnEigRReG5IfVdZklIZzks5rlwwtgaJpZM4Mb45K.

khatchad commented 7 years ago

It's getting stuck on the call to solver.solve(monitor); in com.ibm.wala.ipa.callgraph.propagation.PropagationCallGraphBuilder.makeCallGraph(AnalysisOptions, IProgressMonitor). Let's see if I can get a stack trace.

juliandolby commented 7 years ago

perhaps you have a very large program?

On Mar 14, 2017, at 6:02 PM, Raffi Khatchadourian notifications@github.com wrote:

It's getting stuck on the call to solver.solve(monitor); in com.ibm.wala.ipa.callgraph.propagation.PropagationCallGraphBuilder.makeCallGraph(AnalysisOptions, IProgressMonitor). Let's see if I can get a stack trace.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/wala/WALA/issues/144#issuecomment-286575201, or mute the thread https://github.com/notifications/unsubscribe-auth/ABk3frdU775Co8P5KIDZWu-RhHC3jHe0ks5rlw59gaJpZM4Mb45K.

khatchad commented 7 years ago 
Thread [Worker-6] (Suspended)   
    com.ibm.wala.util.intset.TunedSimpleIntVector(com.ibm.wala.util.intset.SimpleIntVector).get(int) line: 61   
    com.ibm.wala.util.intset.BasicNaturalRelation.getRelated(int) line: 282 
    com.ibm.wala.ipa.callgraph.propagation.PropagationGraph.getStatementsThatUse(com.ibm.wala.ipa.callgraph.propagation.PointsToSetVariable) line: 679  
    com.ibm.wala.ipa.callgraph.propagation.PropagationGraph.getStatementsThatUse(com.ibm.wala.fixpoint.IVariable) line: 1   
    com.ibm.wala.ipa.callgraph.propagation.PropagationSystem(com.ibm.wala.fixedpoint.impl.AbstractFixedPointSolver<T>).changedVariable(T) line: 254 
    com.ibm.wala.ipa.callgraph.propagation.PropagationSystem(com.ibm.wala.fixedpoint.impl.AbstractFixedPointSolver<T>).updateWorkList(com.ibm.wala.fixpoint.AbstractStatement<T,?>) line: 463   
    com.ibm.wala.ipa.callgraph.propagation.PropagationSystem(com.ibm.wala.fixedpoint.impl.AbstractFixedPointSolver<T>).incorporateNewStatement(boolean, boolean, com.ibm.wala.fixpoint.AbstractStatement) line: 301 
    com.ibm.wala.ipa.callgraph.propagation.PropagationSystem(com.ibm.wala.fixedpoint.impl.AbstractFixedPointSolver<T>).newStatement(T, com.ibm.wala.fixpoint.UnaryOperator<T>, T, boolean, boolean) line: 334   
    com.ibm.wala.ipa.callgraph.propagation.PropagationSystem.newConstraint(com.ibm.wala.ipa.callgraph.propagation.PointerKey, com.ibm.wala.fixpoint.UnaryOperator<com.ibm.wala.ipa.callgraph.propagation.PointsToSetVariable>, com.ibm.wala.ipa.callgraph.propagation.PointerKey) line: 346 
    com.ibm.wala.ipa.callgraph.propagation.cfa.nCFABuilder(com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder).processCallingConstraints(com.ibm.wala.ipa.callgraph.CGNode, com.ibm.wala.ssa.SSAAbstractInvokeInstruction, com.ibm.wala.ipa.callgraph.CGNode, com.ibm.wala.ipa.callgraph.propagation.InstanceKey[][], com.ibm.wala.ipa.callgraph.propagation.PointerKey) line: 1659  
    com.ibm.wala.ipa.callgraph.propagation.cfa.nCFABuilder(com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder).processResolvedCall(com.ibm.wala.ipa.callgraph.CGNode, com.ibm.wala.ssa.SSAAbstractInvokeInstruction, com.ibm.wala.ipa.callgraph.CGNode, com.ibm.wala.ipa.callgraph.propagation.InstanceKey[][], com.ibm.wala.ipa.callgraph.propagation.PointerKey) line: 1601    
    com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder.access$2(com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder, com.ibm.wala.ipa.callgraph.CGNode, com.ibm.wala.ssa.SSAAbstractInvokeInstruction, com.ibm.wala.ipa.callgraph.CGNode, com.ibm.wala.ipa.callgraph.propagation.InstanceKey[][], com.ibm.wala.ipa.callgraph.propagation.PointerKey) line: 1579    
    com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$DispatchOperator$1$1.apply(com.ibm.wala.ipa.callgraph.propagation.InstanceKey[]) line: 1755   
    com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$DispatchOperator$1$1.apply(java.lang.Object) line: 1  
    com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$DispatchOperator$1$2(com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$CrossProductRec).rec(int, int) line: 533   
    com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$CrossProductRec$1.act(int) line: 557  
    com.ibm.wala.util.intset.MutableSparseIntSet(com.ibm.wala.util.intset.SparseIntSet).foreach(com.ibm.wala.util.intset.IntSetAction) line: 439    
    com.ibm.wala.util.intset.MutableSharedBitVectorIntSet.foreach(com.ibm.wala.util.intset.IntSetAction) line: 269  
    com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$DispatchOperator$1$2(com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$CrossProductRec).rec(int, int) line: 553   
    com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$DispatchOperator$1$2.<init>(com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$DispatchOperator$1, com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder, com.ibm.wala.ipa.callgraph.propagation.InstanceKey[][], com.ibm.wala.ssa.SSAAbstractInvokeInstruction, com.ibm.wala.ipa.callgraph.CGNode, com.ibm.wala.util.functions.VoidFunction, int, int) line: 1764 
    com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$DispatchOperator$1.act(int) line: 1744    
    com.ibm.wala.util.intset.BitVectorIntSet.actOnWord(com.ibm.wala.util.intset.IntSetAction, int, int) line: 322   
    com.ibm.wala.util.intset.BitVectorIntSet.fastForeachExcluding(com.ibm.wala.util.intset.BitVectorIntSet, com.ibm.wala.util.intset.IntSetAction) line: 308    
    com.ibm.wala.util.intset.BitVectorIntSet.foreachExcluding(com.ibm.wala.util.intset.IntSet, com.ibm.wala.util.intset.IntSetAction) line: 273 
    com.ibm.wala.util.intset.MutableSharedBitVectorIntSet.foreachExcludingInternal(com.ibm.wala.util.intset.MutableSharedBitVectorIntSet, com.ibm.wala.util.intset.IntSetAction) line: 306  
    com.ibm.wala.util.intset.MutableSharedBitVectorIntSet.foreachExcluding(com.ibm.wala.util.intset.IntSet, com.ibm.wala.util.intset.IntSetAction) line: 282    
    com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$DispatchOperator.cpa(com.ibm.wala.ipa.callgraph.propagation.PointsToSetVariable, com.ibm.wala.ipa.callgraph.propagation.PointsToSetVariable[]) line: 1741 
    com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$DispatchOperator.evaluate(com.ibm.wala.ipa.callgraph.propagation.PointsToSetVariable, com.ibm.wala.ipa.callgraph.propagation.PointsToSetVariable[]) line: 1794    
    com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$DispatchOperator.evaluate(com.ibm.wala.fixpoint.IVariable, com.ibm.wala.fixpoint.IVariable[]) line: 1 
    com.ibm.wala.fixedpoint.impl.AbstractFixedPointSolver$Statement(com.ibm.wala.fixedpoint.impl.GeneralStatement<T>).evaluate() line: 37   
    com.ibm.wala.ipa.callgraph.propagation.PropagationSystem(com.ibm.wala.fixedpoint.impl.AbstractFixedPointSolver<T>).incorporateNewStatement(boolean, boolean, com.ibm.wala.fixpoint.AbstractStatement) line: 290 
    com.ibm.wala.ipa.callgraph.propagation.PropagationSystem(com.ibm.wala.fixedpoint.impl.AbstractFixedPointSolver<T>).newStatement(T, com.ibm.wala.fixpoint.AbstractOperator<T>, T[], boolean, boolean) line: 436  
    com.ibm.wala.ipa.callgraph.propagation.PropagationSystem.newSideEffect(com.ibm.wala.fixpoint.AbstractOperator<com.ibm.wala.ipa.callgraph.propagation.PointsToSetVariable>, com.ibm.wala.ipa.callgraph.propagation.PointerKey[]) line: 589   
    com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$ConstraintVisitor.visitInvokeInternal(com.ibm.wala.ssa.SSAAbstractInvokeInstruction, com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$ConstraintVisitor$InvariantComputer) line: 1188    
    com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$ConstraintVisitor.visitInvoke(com.ibm.wala.ssa.SSAInvokeInstruction) line: 1113   
    com.ibm.wala.classLoader.JavaLanguage$JavaInstructionFactory$10(com.ibm.wala.ssa.SSAInvokeInstruction).visit(com.ibm.wala.ssa.SSAInstruction$IVisitor) line: 87 
    com.ibm.wala.ipa.callgraph.propagation.cfa.nCFABuilder(com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder).addBlockInstructionConstraints(com.ibm.wala.ipa.callgraph.CGNode, com.ibm.wala.ssa.IRView, com.ibm.wala.ssa.SSACFG$BasicBlock, com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder$ConstraintVisitor, com.ibm.wala.util.MonitorUtil$IProgressMonitor) line: 292 
    com.ibm.wala.ipa.callgraph.propagation.cfa.nCFABuilder(com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder).addNodeInstructionConstraints(com.ibm.wala.ipa.callgraph.CGNode, com.ibm.wala.util.MonitorUtil$IProgressMonitor) line: 263    
    com.ibm.wala.ipa.callgraph.propagation.cfa.nCFABuilder(com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder).unconditionallyAddConstraintsFromNode(com.ibm.wala.ipa.callgraph.CGNode, com.ibm.wala.util.MonitorUtil$IProgressMonitor) line: 235    
    com.ibm.wala.ipa.callgraph.propagation.cfa.nCFABuilder(com.ibm.wala.ipa.callgraph.propagation.SSAPropagationCallGraphBuilder).addConstraintsFromNode(com.ibm.wala.ipa.callgraph.CGNode, com.ibm.wala.util.MonitorUtil$IProgressMonitor) line: 201   
    com.ibm.wala.ipa.callgraph.propagation.cfa.nCFABuilder(com.ibm.wala.ipa.callgraph.propagation.PropagationCallGraphBuilder).addConstraintsFromNewNodes(com.ibm.wala.util.MonitorUtil$IProgressMonitor) line: 338 
    com.ibm.wala.ipa.callgraph.propagation.StandardSolver.solve(com.ibm.wala.util.MonitorUtil$IProgressMonitor) line: 58    
    com.ibm.wala.ipa.callgraph.propagation.cfa.nCFABuilder(com.ibm.wala.ipa.callgraph.propagation.PropagationCallGraphBuilder).makeCallGraph(com.ibm.wala.ipa.callgraph.AnalysisOptions, com.ibm.wala.util.MonitorUtil$IProgressMonitor) line: 271  
    ...
khatchad commented 7 years ago

I am using the same program above. I do have some local changes to adhere to some recent refactorings. Let me see if that is affecting it somehow.

juliandolby commented 7 years ago

the nCFABuilder is not something we use much. It could be that a large ’n’ plus library code is creating a huge graph…

On Mar 14, 2017, at 6:02 PM, Raffi Khatchadourian notifications@github.com wrote:

It's getting stuck on the call to solver.solve(monitor); in com.ibm.wala.ipa.callgraph.propagation.PropagationCallGraphBuilder.makeCallGraph(AnalysisOptions, IProgressMonitor). Let's see if I can get a stack trace.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/wala/WALA/issues/144#issuecomment-286575201, or mute the thread https://github.com/notifications/unsubscribe-auth/ABk3frdU775Co8P5KIDZWu-RhHC3jHe0ks5rlw59gaJpZM4Mb45K.

khatchad commented 7 years ago

I believe I have set 'n' to 2. Yes, that is possible but I wonder what it is about the collect() call that would do that.

juliandolby commented 7 years ago

Perhaps there is some case with invokeDynamic combining with recursion that the nCFABuilder does not handle… for sanity, can you check that my test works for you if you try to run it?

On Mar 14, 2017, at 6:11 PM, Raffi Khatchadourian notifications@github.com wrote:

I believe I have set 'n' to 2. Yes, that is possible but I wonder what it is about the collect() call that would do that.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/wala/WALA/issues/144#issuecomment-286577387, or mute the thread https://github.com/notifications/unsubscribe-auth/ABk3fmpxQyBnxQ66hhoZAKIZBFUfl1Zhks5rlxCagaJpZM4Mb45K.

khatchad commented 7 years ago

Sure, I'll give it a shot.

khatchad commented 7 years ago

Running mvn clean install now.

khatchad commented 7 years ago

OK here are my test results:

Results :

Tests in error: 
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShapeRhino.testPage3:40 » NoClassDefFound
   »  Unexpected exception, expected<com.ibm.wala.util.WalaException> but was<ja...
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testCrawlPage11:85 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testCrawlPage12:134 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testCrawlPage13:167 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testCrawlPage15:185 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testCrawlPage16:203 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testCrawlPage17:226 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testSkeleton:319 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testPage1:57 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testPage2:68 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testPage4:251 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testList:266 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testNojs:246 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testSkeleton2:335 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testIframeTest2:275 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testWindowOnload:303 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testCrawlPage11b:102 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testWindowx:289 » NoClassDefFound
  TestSimplePageCallGraphShapeRhinoNu>TestSimplePageCallGraphShape.testApolloExample:239 » NoClassDefFound

Tests run: 20, Failures: 0, Errors: 20, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] WALA ............................................... SUCCESS [  0.178 s]
[INFO] targets ............................................ SUCCESS [  0.011 s]
[INFO] e44 ................................................ SUCCESS [  0.227 s]
[INFO] com.ibm.wala.util .................................. SUCCESS [  3.398 s]
[INFO] com.ibm.wala.shrike ................................ SUCCESS [  1.219 s]
[INFO] com.ibm.wala.core .................................. SUCCESS [  2.313 s]
[INFO] com.ibm.wala.cast .................................. SUCCESS [  1.212 s]
[INFO] com.ibm.wala.cast.java ............................. SUCCESS [  0.212 s]
[INFO] com.ibm.wala-feature ............................... SUCCESS [  0.104 s]
[INFO] com.ibm.wala.core.testdata ......................... SUCCESS [  5.436 s]
[INFO] com.ibm.wala.core.tests ............................ SUCCESS [05:43 min]
[INFO] com.ibm.wala.cast.test ............................. SUCCESS [  0.634 s]
[INFO] com.ibm.wala.cast.java.test ........................ SUCCESS [  0.651 s]
[INFO] com.ibm.wala.cast.java.test.data ................... SUCCESS [  0.692 s]
[INFO] com.ibm.wala.cast.java.ecj ......................... SUCCESS [  0.196 s]
[INFO] com.ibm.wala.ide ................................... SUCCESS [  0.318 s]
[INFO] com.ibm.wala.ide-feature ........................... SUCCESS [  0.040 s]
[INFO] com.ibm.wala.ide.tests ............................. SUCCESS [  0.202 s]
[INFO] com.ibm.wala.ide.jdt ............................... SUCCESS [  0.337 s]
[INFO] com.ibm.wala.ide.jdt.test .......................... SUCCESS [01:20 min]
[INFO] com.ibm.wala.cast.js ............................... SUCCESS [  0.647 s]
[INFO] com.ibm.wala.cast.js.rhino ......................... SUCCESS [  0.368 s]
[INFO] com.ibm.wala.cast.js.test.data ..................... SUCCESS [  0.293 s]
[INFO] com.ibm.wala.cast.js.test .......................... SUCCESS [  0.517 s]
[INFO] com.ibm.wala.cast.js.rhino.test .................... SUCCESS [03:28 min]
[INFO] com.ibm.wala.cast.js.html.nu_validator ............. FAILURE [  0.797 s]
[INFO] com.ibm.wala.cast.js.nodejs ........................ SKIPPED
[INFO] com.ibm.wala.cast.js.nodejs.test ................... SKIPPED
[INFO] com.ibm.wala.ide.jsdt .............................. SKIPPED
[INFO] com.ibm.wala.ide.jsdt.tests ........................ SKIPPED
[INFO] com.ibm.wala.dalvik ................................ SKIPPED
[INFO] com.ibm.wala.dalvik.test ........................... SKIPPED
[INFO] com.ibm.wala.scandroid ............................. SKIPPED
[INFO] com.ibm.wala-repository ............................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11:12 min
[INFO] Finished at: 2017-03-14T18:31:41-04:00
[INFO] Final Memory: 113M/1562M
[INFO] ------------------------------------------------------------------------

I only have the Java stuff installed. Those seem to be passing. I am at 7e3066b6cda07ea6e977495b82cb0d2a202bb0c2 with the following local changes:

commit 8fd41c148f5dc36a76e4d7656f0ad4fc91b428d9
Author: Raffi Khatchadourian <raffi.khatchadourian@hunter.cuny.edu>
Date:   Mon Mar 13 16:17:39 2017 -0400

    Use IAnalysisCacheView instead of AnalysisCache.

    It seems that some additional types need to be changed due to
    d24519e97497e24fe8e4495331a649343905694b. This may not be inclusive,
    however.

diff --git a/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/impl/Util.java b/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/impl/Util.java
index a964ad5..9722566 100644
--- a/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/impl/Util.java
+++ b/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/impl/Util.java
@@ -317,7 +317,7 @@ public class Util {
    * @param cha governing class hierarchy
    * @param scope representation of the analysis scope
    */
-  public static CallGraphBuilder makeRTABuilder(AnalysisOptions options, AnalysisCache cache, IClassHierarchy cha,
+  public static CallGraphBuilder makeRTABuilder(AnalysisOptions options, IAnalysisCacheView cache, IClassHierarchy cha,
       AnalysisScope scope) {

     addDefaultSelectors(options, cha);
@@ -432,7 +432,7 @@ public class Util {
    * @return a 0-CFA Call Graph Builder augmented with extra logic for containers
    * @throws IllegalArgumentException if options is null
    */
-  public static SSAPropagationCallGraphBuilder makeZeroContainerCFABuilder(AnalysisOptions options, AnalysisCache cache,
+  public static SSAPropagationCallGraphBuilder makeZeroContainerCFABuilder(AnalysisOptions options, IAnalysisCacheView cache,
       IClassHierarchy cha, AnalysisScope scope) {

     if (options == null) {
@@ -453,12 +453,12 @@ public class Util {
    * @return a 0-1-CFA Call Graph Builder augmented with extra logic for containers
    * @throws IllegalArgumentException if options is null
    */
-  public static SSAPropagationCallGraphBuilder makeZeroOneContainerCFABuilder(AnalysisOptions options, AnalysisCache cache,
+  public static SSAPropagationCallGraphBuilder makeZeroOneContainerCFABuilder(AnalysisOptions options, IAnalysisCacheView cache,
       IClassHierarchy cha, AnalysisScope scope) {
     return makeZeroOneContainerCFABuilder(options, cache, cha, scope, null, null);
   }

-  public static SSAPropagationCallGraphBuilder makeZeroOneContainerCFABuilder(AnalysisOptions options, AnalysisCache cache,
+  public static SSAPropagationCallGraphBuilder makeZeroOneContainerCFABuilder(AnalysisOptions options, IAnalysisCacheView cache,
       IClassHierarchy cha, AnalysisScope scope, ContextSelector appSelector, SSAContextInterpreter appInterpreter) {

     if (options == null) {
diff --git a/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/cfa/ZeroXContainerCFABuilder.java b/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/cfa/ZeroXContainerCFABuilder.java
index 785a159..d18ee90 100644
--- a/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/cfa/ZeroXContainerCFABuilder.java
+++ b/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/cfa/ZeroXContainerCFABuilder.java
@@ -10,9 +10,9 @@
  *******************************************************************************/
 package com.ibm.wala.ipa.callgraph.propagation.cfa;

-import com.ibm.wala.ipa.callgraph.AnalysisCache;
 import com.ibm.wala.ipa.callgraph.AnalysisOptions;
 import com.ibm.wala.ipa.callgraph.ContextSelector;
+import com.ibm.wala.ipa.callgraph.IAnalysisCacheView;
 import com.ibm.wala.ipa.callgraph.impl.DelegatingContextSelector;
 import com.ibm.wala.ipa.callgraph.propagation.SSAContextInterpreter;
 import com.ibm.wala.ipa.cha.IClassHierarchy;
@@ -29,7 +29,7 @@ public class ZeroXContainerCFABuilder extends ZeroXCFABuilder {
    * @param appContextInterpreter application-specific logic to interpret a method in context
    * @throws IllegalArgumentException if options is null
    */
-  public ZeroXContainerCFABuilder(IClassHierarchy cha, AnalysisOptions options, AnalysisCache cache,
+  public ZeroXContainerCFABuilder(IClassHierarchy cha, AnalysisOptions options, IAnalysisCacheView cache,
       ContextSelector appContextSelector, SSAContextInterpreter appContextInterpreter, int instancePolicy) {

     super(cha, options, cache, appContextSelector, appContextInterpreter, instancePolicy);
diff --git a/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/rta/AbstractRTABuilder.java b/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/rta/AbstractRTABuilder.java
index ea68aa6..2d78b3d 100644
--- a/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/rta/AbstractRTABuilder.java
+++ b/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/rta/AbstractRTABuilder.java
@@ -24,6 +24,7 @@ import com.ibm.wala.ipa.callgraph.AnalysisCache;
 import com.ibm.wala.ipa.callgraph.AnalysisOptions;
 import com.ibm.wala.ipa.callgraph.CGNode;
 import com.ibm.wala.ipa.callgraph.ContextSelector;
+import com.ibm.wala.ipa.callgraph.IAnalysisCacheView;
 import com.ibm.wala.ipa.callgraph.impl.DefaultContextSelector;
 import com.ibm.wala.ipa.callgraph.impl.DelegatingContextSelector;
 import com.ibm.wala.ipa.callgraph.impl.Everywhere;
@@ -97,7 +98,7 @@ public abstract class AbstractRTABuilder extends PropagationCallGraphBuilder {
       TypeReference.findOrCreate(ClassLoaderReference.Primordial, "Ljava/lang/ExceptionInInitializerError"),
       TypeReference.findOrCreate(ClassLoaderReference.Primordial, "Ljava/lang/NullPointerException") };

-  protected AbstractRTABuilder(IClassHierarchy cha, AnalysisOptions options, AnalysisCache cache,
+  protected AbstractRTABuilder(IClassHierarchy cha, AnalysisOptions options, IAnalysisCacheView cache,
       ContextSelector appContextSelector, SSAContextInterpreter appContextInterpreter) {
     super(cha, options, cache, new DefaultPointerKeyFactory());
     setInstanceKeys(new ClassBasedInstanceKeys(options, cha));
diff --git a/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/rta/BasicRTABuilder.java b/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/rta/BasicRTABuilder.java
index da30a01..0f3e4bd 100644
--- a/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/rta/BasicRTABuilder.java
+++ b/com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/rta/BasicRTABuilder.java
@@ -23,6 +23,7 @@ import com.ibm.wala.ipa.callgraph.AnalysisCache;
 import com.ibm.wala.ipa.callgraph.AnalysisOptions;
 import com.ibm.wala.ipa.callgraph.CGNode;
 import com.ibm.wala.ipa.callgraph.ContextSelector;
+import com.ibm.wala.ipa.callgraph.IAnalysisCacheView;
 import com.ibm.wala.ipa.callgraph.impl.ExplicitCallGraph;
 import com.ibm.wala.ipa.callgraph.impl.ExplicitCallGraph.ExplicitNode;
 import com.ibm.wala.ipa.callgraph.propagation.InstanceKey;
@@ -41,7 +42,7 @@ import com.ibm.wala.util.intset.MutableIntSet;
  */
 public class BasicRTABuilder extends AbstractRTABuilder {

-  public BasicRTABuilder(IClassHierarchy cha, AnalysisOptions options, AnalysisCache cache, ContextSelector contextSelector,
+  public BasicRTABuilder(IClassHierarchy cha, AnalysisOptions options, IAnalysisCacheView cache, ContextSelector contextSelector,
       SSAContextInterpreter contextInterpreter) {
     super(cha, options, cache, contextSelector, contextInterpreter);
   }
khatchad commented 7 years ago

@juliandolby https://github.com/wala/WALA/issues/144#issuecomment-286590137 should include your test, correct?

juliandolby commented 7 years ago

it does, yes. in LambdaTest, there is testBug144.

On Mar 15, 2017, at 1:03 PM, Raffi Khatchadourian notifications@github.com wrote:

@juliandolby https://github.com/juliandolby #144 (comment) https://github.com/wala/WALA/issues/144#issuecomment-286590137 should include your test, correct?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wala/WALA/issues/144#issuecomment-286810036, or mute the thread https://github.com/notifications/unsubscribe-auth/ABk3foZ3KQ_00Q6ptEDlPtvJpDbzneC4ks5rmBnhgaJpZM4Mb45K.

khatchad commented 7 years ago

Looks like I am experiencing this problem also when I call distinct().

khatchad commented 7 years ago

Perhaps there is some case with invokeDynamic combining with recursion that the nCFABuilder does not handle…

@juliandolby Are there any hints as to where I should be looking to possibly fix this? Is there a (working) implementation in another builder that I could possibly compare with?

msridhar commented 7 years ago

@khatchad what would really help is a reproducible test case against WALA master along with a driver. that way @juliandolby or I could reproduce. if this is a distinct problem from the original issue, maybe we can close and open a new one?

khatchad commented 7 years ago

Thanks, @msridhar. I'll get to work on that. It does seem to be a different issue perhaps specifically involving nCFA.

khatchad commented 7 years ago

Filed #169. Closing this issue as it seems to be fixed.

khatchad commented 7 years ago

@khatchad what would really help is a reproducible test case against WALA master along with a driver. that way @juliandolby or I could reproduce. if this is a distinct problem from the original issue, maybe we can close and open a new one?

@msridhar Thank you again for your help with this issue. Does the information in #169 suffice for what you are looking for? I figured that you can always switch the remote origin master to point at the main WALA repository if you need an update. If not, please let me know and I will give it to you in some other form. Thanks, and have a great weekend!

msridhar commented 7 years ago

@khatchad I've commented on #169