search

vitruv-tools / Vitruv

View-based Development and Model Consistency Framework
http://vitruv.tools
Eclipse Public License 1.0
14 stars 19 forks source link

Eclipse URI resolver may resolve pathmaps across tests #464

Closed JanWittler closed 1 year ago

JanWittler commented 3 years ago

I am currently experiencing test failures depending on the order in which I execute tests. In particular, if a certain test is executed, all tests after it are failing. If those tests are executed independently, they succeed. It seems that the ResourceRepository is not correctly cleaned up. Attached you can see the stack trace. The critical test propagates changes where Interface.java is created. This class is not present in the other tests, however the test fails because of a pathmap to it thus it must be an artifact of the previous test. The correct behaviour should be that the entire VSUM state is reset before a new test is executed.

Stacktrace java.lang.IllegalStateException: Could not delete URI pathmap:/javaclass/basic.config.Interface.java at tools.vitruv.framework.vsum.ModelInstance.delete(ModelInstance.java:106) at tools.vitruv.framework.vsum.repositories.ResourceRepositoryImpl.saveOrDeleteModels(ResourceRepositoryImpl.java:160) at tools.vitruv.framework.vsum.VirtualModelImpl.save(VirtualModelImpl.java:86) at tools.vitruv.framework.vsum.VirtualModelImpl.propagateChange(VirtualModelImpl.java:98) at tools.vitruv.testutils.ChangePublishingTestView.lambda$0(ChangePublishingTestView.java:89) at tools.vitruv.testutils.ChangePublishingTestView.lambda$5(ChangePublishingTestView.java:156) at edu.kit.ipd.sdq.commons.util.java.lang.IterableUtil.flatMapFixedTo(IterableUtil.java:60) at edu.kit.ipd.sdq.commons.util.java.lang.IterableUtil.flatMapFixed(IterableUtil.java:55) at tools.vitruv.testutils.ChangePublishingTestView.propagateChanges(ChangePublishingTestView.java:158) at tools.vitruv.testutils.ChangePublishingTestView.propagate(ChangePublishingTestView.java:121) at tools.vitruv.testutils.VitruvApplicationTest.propagate(VitruvApplicationTest.java:132) at tools.vitruv.applications.external.umljava.tests.uml2java.advancedsuite.AdvancedSuiteTest.enrichJavaModel(AdvancedSuiteTest.java:144) at tools.vitruv.applications.external.umljava.tests.uml2java.Uml2JavaStateBasedChangeTest.preloadModel(Uml2JavaStateBasedChangeTest.java:51) at tools.vitruv.applications.external.umljava.tests.uml2java.advancedsuite.AdvancedSuiteTest.preloadModel(AdvancedSuiteTest.java:49) at tools.vitruv.applications.external.umljava.tests.uml2java.StateBasedChangeTest.setup(StateBasedChangeTest.java:98) at tools.vitruv.applications.external.umljava.tests.uml2java.StateBasedChangeDifferencesTest.setup(StateBasedChangeDifferencesTest.java:46) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688) at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptLifecycleMethod(TimeoutExtension.java:126) at org.junit.jupiter.engine.extension.TimeoutExtension.interceptBeforeEachMethod(TimeoutExtension.java:76) at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115) at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105) at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104) at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98) at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeMethodInExtensionContext(ClassBasedTestDescriptor.java:490) at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$synthesizeBeforeEachMethodAdapter$19(ClassBasedTestDescriptor.java:475) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeEachMethods$2(TestMethodTestDescriptor.java:167) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeMethodsOrCallbacksUntilExceptionOccurs$5(TestMethodTestDescriptor.java:195) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeMethodsOrCallbacksUntilExceptionOccurs(TestMethodTestDescriptor.java:195) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeEachMethods(TestMethodTestDescriptor.java:164) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:127) at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:65) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129) at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127) at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126) at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84) at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32) at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:84) at org.eclipse.jdt.internal.junit5.runner.JUnit5TestReference.run(JUnit5TestReference.java:98) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:41) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:542) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:770) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:464) at org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner.main(RemotePluginTestRunner.java:117) at org.eclipse.pde.internal.junit.runtime.PlatformUITestHarness.lambda$0(PlatformUITestHarness.java:45) at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40) at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:185) at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4181) at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3841) at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$5.run(PartRenderingEngine.java:1157) at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338) at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1046) at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:155) at org.eclipse.ui.internal.Workbench.lambda$3(Workbench.java:644) at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338) at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:551) at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:156) at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:152) at org.eclipse.pde.internal.junit.runtime.NonUIThreadTestApplication.runApp(NonUIThreadTestApplication.java:53) at org.eclipse.pde.internal.junit.runtime.UITestApplication.runApp(UITestApplication.java:47) at org.eclipse.pde.internal.junit.runtime.NonUIThreadTestApplication.start(NonUIThreadTestApplication.java:48) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:401) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:653) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:590) at org.eclipse.equinox.launcher.Main.run(Main.java:1461) at org.eclipse.equinox.launcher.Main.main(Main.java:1434) Caused by: java.net.MalformedURLException: unknown protocol: pathmap at java.base/java.net.URL.(URL.java:652) at java.base/java.net.URL.(URL.java:541) at java.base/java.net.URL.(URL.java:488) at org.eclipse.emf.ecore.resource.impl.URIHandlerImpl.delete(URIHandlerImpl.java:228) at org.eclipse.emf.ecore.resource.impl.ExtensibleURIConverterImpl.delete(ExtensibleURIConverterImpl.java:364) at org.eclipse.emf.ecore.resource.impl.ResourceImpl.delete(ResourceImpl.java:1737) at tools.vitruv.framework.domains.repository.DomainAwareResource.delete(DomainAwareResource.java:179) at tools.vitruv.framework.vsum.ModelInstance.delete(ModelInstance.java:93) ... 116 more
JanWittler commented 3 years ago

It seems that the issue is local to some test package. When running an entire suite, subsequent tests only fail for test cases within the same package of the problematic test. When entering another package later, tests succeed again.

JanWittler commented 3 years ago

The issue is caused by how Eclipse resolves pathmap URIs. When the resource to which the pathmap points is not yet loaded in the local resource set, the URI is resolved by asking the URIMappingRegistryImpl.INSTANCE. All paths to loaded resources are backed up there. In my case the issue was caused because in the second test case I was loading a resource that referenced another resource. That referenced resource was not yet loaded in the second test thus there was no entry in the mapping registry with the correct path. However, in the first test a resource with the same name was loaded. Thus the mapping registry contained an entry with the path pointing to the test output of the first test which was then loaded in the second test and caused the crash.

For my test suite I was able to solve the test by manually loading the incorrectly resolved resource before loading the resource that has the reference. However, in general there should never be a dependency on the order in which tests are executed. I am not completely sure whether the resolving error is an issue of Eclipse or rather JaMoPP (as this is known to have loading issues Vitruv-Domains#97).

jGleitz commented 3 years ago

Thus the mapping registry contained an entry with the path pointing to the test output of the first test

In other places of Vitruv, we assume that pathmap resources are effectively read-only. What kind of reference was using a pathmap URI to reference a not read-only resource?

JanWittler commented 3 years ago

You get that behaviour by loading a Java class that references another custom Java class (i.e. as a field type or method argument type). When loading such a class, JaMoPP uses pathmap to reference those classes.

Thus the mapping registry contained an entry with the path pointing to the test output of the first test

To be more precise: it contained an entry pointing to the Java class file at [test_dir]/src/...

HeikoKlare commented 3 years ago

Since JaMoPP caches classes and thus may resolve classes created in other tests, we usually reset the classpath for each test case (see, for example, the Java to PCM tests. Potentially, it would be more reasonable to cleanup after the test rather than before, but the effect should be the same. Can you try whether that cleanup resolves the issue in your tests?

JanWittler commented 3 years ago

Thanks for the hint. However, the linked code does not change the behaviour since the problem is not the JaMoPP cache but the URIMappingRegistryImpl.INSTANCE cache.

HeikoKlare commented 3 years ago

I actually expected every registry caching some of the JaMoPP classpath elements to be cleaned up when resetting the classpath, but that does not seem to be the case :-/ Would it be possible to cleanup the URIMappingRegistryImpl.INSTANCE cache by hand as well?

I did not know about that behavior, which also seems to restrict the productive usability of JaMoPP, because having multiple projects with the same classes in the same packages would be problematic.

JanWittler commented 3 years ago

I tried completely resetting the URIMappingRegistryImpl.INSTANCE but this resulted in a failure since no pathmaps at all could be resolved anymore. We could try to create some filter to only reset pathmaps created by the test case but I assume this to be very error-prone. The pathmap resolving is based on some delegation mechanism, i.e. if the current resolver does not know the pathmap, the parent is asked until eventually the root, which is the URIMappingRegistryImpl.INSTANCE, is asked. I guess it was possible to insert some custom resolver in that chain which gets reset after each test run. However, I did not test this.