search

ponder-lab / Optimize-Java-8-Streams-Refactoring

Refactorings for optimizing Java 8 stream client code for greater parallelism and efficiency.
http://cuny.is/streams
Eclipse Public License 1.0
8 stars 7 forks source link

Exception thrown for Arrays.stream() #80

Closed khatchad closed 6 years ago

khatchad commented 7 years ago 
-- Error Log from JUnit --
Class: edu.cuny.hunter.streamrefactoring.ui.tests.ConvertStreamToParallelRefactoringTest
Method: testArraysStream
Actual: null
Expected: null
Stack Trace:
java.lang.IllegalArgumentException: null pa 
    at com.ibm.wala.analysis.pointers.HeapGraphImpl.<init>(HeapGraphImpl.java:43)
    at com.ibm.wala.analysis.pointers.BasicHeapGraph.<init>(BasicHeapGraph.java:74)
    at com.ibm.wala.client.AbstractAnalysisEngine.getHeapGraph(AbstractAnalysisEngine.java:270)
    at edu.cuny.hunter.streamrefactoring.core.analysis.StreamStateMachine.start(StreamStateMachine.java:256)
    at edu.cuny.hunter.streamrefactoring.core.analysis.Stream.<init>(Stream.java:205)
    at edu.cuny.hunter.streamrefactoring.core.analysis.StreamAnalysisVisitor.visit(StreamAnalysisVisitor.java:49)
    at org.eclipse.jdt.core.dom.MethodInvocation.accept0(MethodInvocation.java:231)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2711)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChild(ASTNode.java:2759)
    at org.eclipse.jdt.core.dom.ExpressionStatement.accept0(ExpressionStatement.java:145)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2711)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2782)
    at org.eclipse.jdt.core.dom.Block.accept0(Block.java:137)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2711)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChild(ASTNode.java:2759)
    at org.eclipse.jdt.core.dom.MethodDeclaration.accept0(MethodDeclaration.java:635)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2711)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2782)
    at org.eclipse.jdt.core.dom.TypeDeclaration.accept0(TypeDeclaration.java:470)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2711)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2782)
    at org.eclipse.jdt.core.dom.CompilationUnit.accept0(CompilationUnit.java:212)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2711)
    at edu.cuny.hunter.streamrefactoring.ui.tests.ConvertStreamToParallelRefactoringTest.helper(ConvertStreamToParallelRefactoringTest.java:310)
    at edu.cuny.hunter.streamrefactoring.ui.tests.ConvertStreamToParallelRefactoringTest.testArraysStream(ConvertStreamToParallelRefactoringTest.java:279)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at junit.framework.TestCase.runTest(TestCase.java:176)
    at junit.framework.TestCase.runBare(TestCase.java:141)
    at junit.framework.TestResult$1.protect(TestResult.java:122)
    at junit.framework.TestResult.runProtected(TestResult.java:142)
    at junit.framework.TestResult.run(TestResult.java:125)
    at junit.framework.TestCase.run(TestCase.java:129)
    at junit.extensions.TestDecorator.basicRun(TestDecorator.java:23)
    at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
    at junit.framework.TestResult.runProtected(TestResult.java:142)
    at junit.extensions.TestSetup.run(TestSetup.java:27)
    at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:121)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner.main(RemotePluginTestRunner.java:66)
    at org.eclipse.pde.internal.junit.runtime.PlatformUITestHarness.lambda$0(PlatformUITestHarness.java:43)
    at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
    at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:182)
    at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4536)
    at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4154)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$4.run(PartRenderingEngine.java:1121)
    at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:336)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1022)
    at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:150)
    at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:693)
    at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:336)
    at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:610)
    at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:148)
    at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:138)
    at org.eclipse.pde.internal.junit.runtime.NonUIThreadTestApplication.runApp(NonUIThreadTestApplication.java:52)
    at org.eclipse.pde.internal.junit.runtime.UITestApplication.runApp(UITestApplication.java:43)
    at org.eclipse.pde.internal.junit.runtime.NonUIThreadTestApplication.start(NonUIThreadTestApplication.java:46)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:388)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:243)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:673)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:610)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1519)
    at org.eclipse.equinox.launcher.Main.main(Main.java:1492)
khatchad commented 7 years ago

@saledouble This one seems very much related to #67 but is still failing it seems. Can you have a look at this one next? Thanks so much.

yiming-tang-cs commented 7 years ago

The current exception:

!ENTRY org.eclipse.jdt.ui 4 10001 2017-11-05 22:08:39.730
!MESSAGE Internal Error
!STACK 0
java.lang.IllegalArgumentException: Can't find instance key for: Stream [streamCreation=Arrays.stream(new Object[1]), enclosingMethodDeclaration=m, possibleExecutionModes=[SEQUENTIAL], possibleOrderings=[null], status=0] using tracked instances: [SITE_IN_NODE{< Primordial, Ljava/util/stream/StreamSupport, stream(Ljava/util/Spliterator;Z)Ljava/util/stream/Stream; >:NEW <Primordial,Ljava/util/stream/ReferencePipeline$Head>@5 in CallStringContext: [ java.util.Arrays.stream([Ljava/lang/Object;II)Ljava/util/stream/Stream;@7 java.util.Arrays.stream([Ljava/lang/Object;)Ljava/util/stream/Stream;@4 ]}]
    at edu.cuny.hunter.streamrefactoring.core.analysis.Stream.lambda$5(Stream.java:640)
    at java.util.Optional.orElseThrow(Optional.java:290)
    at edu.cuny.hunter.streamrefactoring.core.analysis.Stream.getInstanceKey(Stream.java:640)
    at edu.cuny.hunter.streamrefactoring.core.analysis.StreamStateMachine.start(StreamStateMachine.java:474)
    at edu.cuny.hunter.streamrefactoring.core.analysis.Stream.<init>(Stream.java:206)
    at edu.cuny.hunter.streamrefactoring.core.analysis.StreamAnalysisVisitor.visit(StreamAnalysisVisitor.java:51)
    at org.eclipse.jdt.core.dom.MethodInvocation.accept0(MethodInvocation.java:231)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2711)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChild(ASTNode.java:2759)
    at org.eclipse.jdt.core.dom.ExpressionStatement.accept0(ExpressionStatement.java:145)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2711)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2782)
    at org.eclipse.jdt.core.dom.Block.accept0(Block.java:137)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2711)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChild(ASTNode.java:2759)
    at org.eclipse.jdt.core.dom.MethodDeclaration.accept0(MethodDeclaration.java:635)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2711)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2782)
    at org.eclipse.jdt.core.dom.TypeDeclaration.accept0(TypeDeclaration.java:470)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2711)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2782)
    at org.eclipse.jdt.core.dom.CompilationUnit.accept0(CompilationUnit.java:212)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2711)
    at edu.cuny.hunter.streamrefactoring.core.refactorings.ConvertToParallelStreamRefactoringProcessor.checkFinalConditions(ConvertToParallelStreamRefactoringProcessor.java:191)
    at org.eclipse.ltk.core.refactoring.participants.ProcessorBasedRefactoring.checkFinalConditions(ProcessorBasedRefactoring.java:222)
    at org.eclipse.ltk.core.refactoring.Refactoring.checkAllConditions(Refactoring.java:162)
    at edu.cuny.hunter.streamrefactoring.eval.handlers.EvaluateConvertToParallelStreamRefactoringHandler.lambda$0(EvaluateConvertToParallelStreamRefactoringHandler.java:145)
    at org.eclipse.core.runtime.jobs.Job$1.run(Job.java:161)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:56)
yiming-tang-cs commented 7 years ago

The exception is thrown by: https://github.com/ponder-lab/Java-8-Stream-Refactoring/blob/0962c05e807a47c676d096e9ebe207df8c1f8f9e/edu.cuny.hunter.streamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/Stream.java#L634-L642 The problem is from the method instanceKeyCorrespondsWithInstantiationInstruction: https://github.com/ponder-lab/Java-8-Stream-Refactoring/blob/0962c05e807a47c676d096e9ebe207df8c1f8f9e/edu.cuny.hunter.streamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/safe/Util.java#L58 The value of callSiteReference:

invokestatic < Primordial, Ljava/util/stream/StreamSupport, stream(Ljava/util/Spliterator;Z)Ljava/util/stream/Stream; >@7
invokestatic < Primordial, Ljava/util/Arrays, stream([Ljava/lang/Object;II)Ljava/util/stream/Stream; >@4

The value of instruction.getCallSite():

invokestatic < Application, Ljava/util/Arrays, stream([Ljava/lang/Object;)Ljava/util/stream/Stream; >@4
yiming-tang-cs commented 7 years ago

Ah, I mentioned this issue in #67

khatchad commented 7 years ago

I don't think that node corresponds to the call in question per say. clinit stands for "class init" I believe.

khatchad commented 6 years ago

This seems related to #98. Perhaps the reason that there's a null PA is because the instance key isn't found.

khatchad commented 6 years ago

Actually, for the Arrays.stream() test case, I am seeing the following:

WARNING: Encountered probable unhandled case while processing: Arrays.stream(new Object[1])
edu.cuny.hunter.streamrefactoring.core.analysis.InstanceKeyNotFoundException: ...

It continues:

Can't find instance key for: Arrays.stream(new Object[1]) using tracked instances: [SITE_IN_NODE{< Primordial, Ljava/util/stream/StreamSupport, stream(Ljava/util/Spliterator;Z)Ljava/util/stream/Stream; >:NEW <Primordial,Ljava/util/stream/ReferencePipeline$Head>@5 in CallStringContext: [ java.util.Arrays.stream([Ljava/lang/Object;II)Ljava/util/stream/Stream;@7 java.util.Arrays.stream([Ljava/lang/Object;)Ljava/util/stream/Stream;@4 ]}]

And the stack trace:

    at edu.cuny.hunter.streamrefactoring.core.analysis.Stream.lambda$5(Stream.java:675)
    at java.util.Optional.orElseThrow(Optional.java:290)
    at edu.cuny.hunter.streamrefactoring.core.analysis.Stream.getInstanceKey(Stream.java:675)
    at edu.cuny.hunter.streamrefactoring.core.analysis.StreamStateMachine.start(StreamStateMachine.java:474)
    at edu.cuny.hunter.streamrefactoring.core.analysis.Stream.<init>(Stream.java:214)
    at edu.cuny.hunter.streamrefactoring.core.analysis.StreamAnalysisVisitor.visit(StreamAnalysisVisitor.java:51)
    at org.eclipse.jdt.core.dom.MethodInvocation.accept0(MethodInvocation.java:231)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2796)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChild(ASTNode.java:2844)
    at org.eclipse.jdt.core.dom.ExpressionStatement.accept0(ExpressionStatement.java:145)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2796)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2867)
    at org.eclipse.jdt.core.dom.Block.accept0(Block.java:137)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2796)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChild(ASTNode.java:2844)
    at org.eclipse.jdt.core.dom.MethodDeclaration.accept0(MethodDeclaration.java:635)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2796)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2867)
    at org.eclipse.jdt.core.dom.TypeDeclaration.accept0(TypeDeclaration.java:470)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2796)
    at org.eclipse.jdt.core.dom.ASTNode.acceptChildren(ASTNode.java:2867)
    at org.eclipse.jdt.core.dom.CompilationUnit.accept0(CompilationUnit.java:260)
    at org.eclipse.jdt.core.dom.ASTNode.accept(ASTNode.java:2796)
    at edu.cuny.hunter.streamrefactoring.ui.tests.ConvertStreamToParallelRefactoringTest.helper(ConvertStreamToParallelRefactoringTest.java:258)
    at edu.cuny.hunter.streamrefactoring.ui.tests.ConvertStreamToParallelRefactoringTest.testArraysStream(ConvertStreamToParallelRefactoringTest.java:359)
...
khatchad commented 6 years ago

So it looks like the instance key for Arrays.stream(new Object[1]) (whatever that is?) doesn't match the instance key containing stream(Ljava/util/Spliterator;Z)Ljava/util/stream/Stream;.

yiming-tang-cs commented 6 years ago

This seems related to #98. Perhaps the reason that there's a null PA is because the instance key isn't foun

Yes, it is related to #98.

The current type of classloader for Arrays is Application (the right one should be Primordial?), which is gotten from method buildCallGraph().

https://github.com/ponder-lab/Java-8-Stream-Refactoring/blob/312334fe6047b09db40e5d551c7aff8861fe46eb/edu.cuny.hunter.streamrefactoring.core/src/edu/cuny/hunter/streamrefactoring/core/analysis/Stream.java#L764-L788

It is clear that the problem is from entry point.

khatchad commented 6 years ago

If the wrong classloader is being used for Arrays, that doesn't sound related to #98 at all. Arrays is a library method, so it should not be in Application.

khatchad commented 6 years ago

I am actually not convinced that it's related to #98 by looking at the test case in question: https://github.com/ponder-lab/Java-8-Stream-Refactoring/blob/312334fe6047b09db40e5d551c7aff8861fe46eb/edu.cuny.hunter.streamrefactoring.tests/resources/ConvertStreamToParallel/testArraysStream/in/A.java#L5-L9

98 happens when the collection used to derive the stream is instantiated in a different method. We're not using a collection here (but rather an array), but the array is instantiated in the same method.

khatchad commented 6 years ago

In fact, in this example, there is only one method and it happens to be the "declaring" method, i.e., the method that instantiates the stream. As such, given the current implementation, it will be set as the entry point.

yiming-tang-cs commented 6 years ago

the array is instantiated in the same method

It seems that the array is not instantiated. The test case just calls the static method of class Arrays. Did you mean new Object[1] in the test case?

khatchad commented 6 years ago

It seems that the array is not instantiated.

What makes you feel that way?

Did you mean new Object[1] in the test case?

Indeed, that is an array instantiation.

khatchad commented 6 years ago

I would think that these two call site references should match:

invokestatic < Primordial, Ljava/util/Arrays, stream([Ljava/lang/Object;II)Ljava/util/stream/Stream; >@4
invokestatic < Application, Ljava/util/Arrays, stream([Ljava/lang/Object;)Ljava/util/stream/Stream; >@4

I wonder what the II means.

khatchad commented 6 years ago

Having trouble reproducing this.

khatchad commented 6 years ago

I think a problem here is that N=2 is not enough for this case.

khatchad commented 6 years ago

Related to #176.

khatchad commented 6 years ago

Ran into a snag here. There's some code in WALA that won't use the given length under certain conditions, so we can't increase the call string length arbitrarily. Looking into why those conditions are there.

khatchad commented 6 years ago

OK, it looks like WALA is trying to build a call string with the lenght we give it but it is an incremental process. It looks like in one our test cases, it just never gets up to this length. Not sure why. The result is that we don't get up to the client code.

I wonder how this test was passing previously ...

khatchad commented 6 years ago

Looks like ctors return void and we are asking where ctors that construct streams implement base stream by examining their return type (void), which, of course, is false. If it's a ctor, then we need to use the declaring class.