[tests] Migrate to life support

[eclipse-modisco-bot]

| --- | --- | | Bugzilla Link | 552989 | | Status | NEW | | Importance | P3 normal | | Reported | Nov 13, 2019 05:25 EDT | | Modified | Jan 16, 2020 11:34 EDT | | Depends on | 553277 | | See also | 552988, 415657, Gerrit change https://git.eclipse.org/r/153331, Gerrit change https://git.eclipse.org/r/153344, Gerrit change https://git.eclipse.org/r/153348, Git commit f7e2dd76, Git commit 7f9310e5, 559115 | | Reporter | Ed Willink |

Description

Rejuvenating Modisco in accordance with Bug 552988 starts with quite a few test failures for MoDisco_AllTestsInUIThread (Fast) launch (MoDisco_AllTestsInUIThread (Fast) with skip.long.junit.tests set true) and with a skip.long.junit.tests added to testJspFileExistence which times out.

The tests take about 4 minutes to run:\ 195 tests 12 skipped 8 errors 7 failures

Many of the problems look as if they may just be extra problem markers perhaps from the newish EMF EAnnotation Validation.

This bug tracks the initial brute force SKIP of all awkward tests in order to get a release built (and hopefully the subsequent resolution).

[eclipse-modisco-bot]

By Ed Willink on Nov 14, 2019 05:49

The origin/emffacetreleng-auto-pom branch contains the auto-generated pom.xml.

Most of the pom.xml are seriously boring, however for the Acceleo plugins there are extra steps to drive the Acceleo maven plugins. These may make some disabled tests work.

[eclipse-modisco-bot]

By Ed Willink on Nov 14, 2019 07:54

Many of the tests have major stack traces in the console log but seem to pass anyway.

org.eclipse.modisco.jee.jsp.discoverer.tests seems to fill the work folder recursively causing Eclipse to hang when refreshing.

[eclipse-modisco-bot]

By Ed Willink on Nov 17, 2019 06:00

Many of the tests have major stack traces in the console log but seem to pass anyway.

org.eclipse.modisco.jee.jsp.discoverer.tests seems to fill the work folder recursively causing Eclipse to hang when refreshing. Really evil, the depth of the directory tree prevents Windows deleting it even when the Recycle Bin is bypassed.

Once the appprently 'successful' build is attempted on Jenkins there are 35 failures. Ignore them too for now. Most of the errors seem to be of the form:

!ENTRY org.eclipse.jdt.core 4 2 2019-11-17 08:42:28.343\ !MESSAGE Problems occurred when invoking code from plug-in: "org.eclipse.jdt.core".\ !STACK 0\ java.lang.UnsupportedOperationException\ at java.util.AbstractMap.put(AbstractMap.java:209)\ at org.eclipse.pde.internal.core.PluginModelManager.handleAdd(PluginModelManager.java:870)\ at org.eclipse.pde.internal.core.PluginModelManager.modelsChanged(PluginModelManager.java:261)

Where the AbstractMap.put UnsupportedOperationException seems to suggest that JDT/PDE created a wrong kind of Map. Very odd.

[eclipse-modisco-bot]

By Ed Willink on Nov 17, 2019 06:50

(In reply to Ed Willink from comment #3)

Ignore them too for now.

By using the JUnit @Ignore // FIXME Bug 552989

But beware the test pom.xml have the not-recommended

<testFailureIgnore>true</testFailureIgnore>

although strangely some failures seem to fail the build anyway.

[eclipse-modisco-bot]

By Ed Willink on Nov 17, 2019 06:56

Eventually build passes after 51 tests skipped (@Ignore) leaving 100 tests passing; much better than nothing. Probably only a couple of fixes to get the skipped tests going.

[eclipse-modisco-bot]

By Ed Willink on Nov 20, 2019 10:37

(In reply to Ed Willink from comment #5)

Probably only a couple of fixes to get the skipped tests going.

See Bug 553277. The use of J2SE-1.5 may well be the cause of many failures. (QVTd's auto-generated projects use JavaSE-1.8.)

[eclipse-modisco-bot]

By Ed Willink on Nov 21, 2019 04:08

Changing to Java 8 seems to make zero difference.

Some tests still only fail if previous tests have run others such as FacetTests.test001 fail all by themselves; seemingly with many more than the expected 6 problems (probably deprecation warnings).

Tweaking the test to wait flushing events on the UI thread enables the failing project to be played with interactively; it absolutely refuses to build since org.eclipse.core.runtime is not found even though it is in the configurtaion.

Impossible. Ah! There is a perverse target platform definition with nothing useful in it.

[eclipse-modisco-bot]

By Ed Willink on Nov 21, 2019 04:14

(In reply to Ed Willink from comment #7)

There is a perverse target platform definition with nothing useful in it.

The build has been Helios based, so it looks as if the target platform may have never been upgraded to e4!

[eclipse-modisco-bot]

By Ed Willink on Nov 21, 2019 05:16

Trying to understnad the target platform definition init reveals crazy reflective code doing seemingly crazy things in a way that has perhaps fallen over a recent platform deprecation.

Is a target platform definition a good idea?

• pro: more conistent source of completion test/marker problems/EMF registrations
• con: testing an obsolete platform that is not what the user experiences

Hm! do we try to mend crazy broken code or cope with the ripplres of changing policy?

Eliminating the target pltform definition enables the broken test001 to work.

Let's try to lose the spurious target platform.

[eclipse-modisco-bot]

By Ed Willink on Nov 23, 2019 05:11

(In reply to Ed Willink from comment #9)

Let's try to lose the spurious target platform.

The Tycho build run locally is now pretty promising. Almost identical to the prevailing state.

But the JIRO Tycho build has about 37 tests that fail. Most are in org.eclipse.modisco.infra.query.tests.UnitTests where 75% fail, 25% pass.

If it was a total failure or some of the more challenging tests, one might blame the SWTbot setup or ... but some failures seem to be comparatively boring. Only slight hint is that maybe the failures correlate with problem markers.

The immediate stack trace is:

java.lang.AssertionError: \ Wrong number of markers. Expected 1, got 0:

at org.eclipse.modisco.infra.query.tests.UnitTests.wrongX(UnitTests.java:550)\
at org.eclipse.modisco.infra.query.tests.UnitTests.wrongX(UnitTests.java:520)\
at org.eclipse.modisco.infra.query.tests.UnitTests.wrongScope(UnitTests.java:461)

Multiple occurences follow of:

!ENTRY org.eclipse.jdt.core 4 2 2019-11-22 17:44:52.148\ !MESSAGE Problems occurred when invoking code from plug-in: "org.eclipse.jdt.core".\ !STACK 0\ java.lang.UnsupportedOperationException\ at java.util.AbstractMap.put(AbstractMap.java:209)\ at org.eclipse.pde.internal.core.PluginModelManager.handleAdd(PluginModelManager.java:870)\ at org.eclipse.pde.internal.core.PluginModelManager.modelsChanged(PluginModelManager.java:261)\ at org.eclipse.pde.internal.core.AbstractModelManager.fireModelProviderEvent(AbstractModelManager.java:37)\ at org.eclipse.pde.internal.core.WorkspaceModelManager.createAndFireEvent(WorkspaceModelManager.java:279)\ at org.eclipse.pde.internal.core.WorkspacePluginModelManager.createAndFireEvent(WorkspacePluginModelManager.java:576)\ at org.eclipse.pde.internal.core.WorkspaceModelManager.processModelChanges(WorkspaceModelManager.java:258)\ at org.eclipse.pde.internal.core.WorkspaceModelManager.processModelChanges(WorkspaceModelManager.java:216)\ at org.eclipse.pde.internal.core.WorkspacePluginModelManager.processModelChanges(WorkspacePluginModelManager.java:565)\ at org.eclipse.pde.internal.core.WorkspaceModelManager.resourceChanged(WorkspaceModelManager.java:134)\ at org.eclipse.jdt.internal.core.DeltaProcessingState$1.run(DeltaProcessingState.java:472)\ at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)\ at org.eclipse.jdt.internal.core.DeltaProcessingState.resourceChanged(DeltaProcessingState.java:465)\ at org.eclipse.core.internal.events.NotificationManager$1.run(NotificationManager.java:305)\ at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)\ at org.eclipse.core.internal.events.NotificationManager.notify(NotificationManager.java:295)\ at org.eclipse.core.internal.events.NotificationManager.broadcastChanges(NotificationManager.java:158)\ at org.eclipse.core.internal.resources.Workspace.broadcastPostChange(Workspace.java:379)\ at org.eclipse.core.internal.resources.Workspace.endOperation(Workspace.java:1502)\ at org.eclipse.core.internal.resources.Project.create(Project.java:311)\ at org.eclipse.core.internal.resources.Project.create(Project.java:251)\ at org.eclipse.core.internal.resources.Project.create(Project.java:246)\ at org.eclipse.gmt.modisco.infra.common.core.internal.utils.ProjectUtils.createPluginProject(ProjectUtils.java:428)\ at org.eclipse.gmt.modisco.infra.common.core.internal.utils.ProjectUtils.create(ProjectUtils.java:413)\ at org.eclipse.gmt.modisco.infra.common.core.internal.utils.ProjectUtils.createTestProject(ProjectUtils.java:467)\ at org.eclipse.gmt.modisco.infra.common.core.internal.utils.ProjectUtils.createTestProject(ProjectUtils.java:446)\ at org.eclipse.modisco.infra.query.tests.Utils.createProject(Utils.java:40)\ at org.eclipse.modisco.infra.query.tests.UnitTests.wrongX(UnitTests.java:531)

And ultimately:

!ENTRY org.eclipse.core.jobs 4 2 2019-11-22 17:44:52.346\ !MESSAGE An internal error occurred during: "Building workspace".\ !STACK 0\ java.lang.UnsupportedOperationException: Not able to create StateObjectFactory implementation: org.eclipse.osgi.internal.resolver.StateObjectFactoryImpl\ at org.eclipse.osgi.service.resolver.StateObjectFactory$StateObjectFactoryProxy.getImplementation(StateObjectFactory.java:510)\ at org.eclipse.osgi.service.resolver.StateObjectFactory$StateObjectFactoryProxy.createState(StateObjectFactory.java:525)\ at org.eclipse.pde.internal.core.DynamicPluginProjectReferences.getDependentProjects(DynamicPluginProjectReferences.java:53)\ at org.eclipse.core.internal.resources.ProjectDescription.computeDynamicReferencesForProject(ProjectDescription.java:950)\ at org.eclipse.core.internal.resources.ProjectDescription.getAllBuildConfigReferences(ProjectDescription.java:266)\ at org.eclipse.core.internal.resources.Project.internalGetReferencedBuildConfigs(Project.java:787)\ at org.eclipse.core.internal.resources.Workspace.computeActiveBuildConfigGraph(Workspace.java:755)\ at org.eclipse.core.internal.resources.Workspace.getBuildGraph(Workspace.java:1620)\ at org.eclipse.core.internal.resources.Workspace.getBuildOrder(Workspace.java:1601)\ at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:154)\ at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:244)\ at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)\ Caused by: java.lang.ClassNotFoundException: org.eclipse.osgi.internal.resolver.StateObjectFactoryImpl\ at java.net.URLClassLoader.findClass(URLClassLoader.java:382)\ at java.lang.ClassLoader.loadClass(ClassLoader.java:424)\ at java.lang.ClassLoader.loadClass(ClassLoader.java:357)\ at java.lang.Class.forName0(Native Method)\ at java.lang.Class.forName(Class.java:264)\ at org.eclipse.osgi.service.resolver.StateObjectFactory$StateObjectFactoryProxy.getImplementation(StateObjectFactory.java:507) ... 11 more

The repeated createProject entry culd be Tycho's fusineess about modifying readonly locations, but it should be the same locally.

The StateObjectFactory create failure suggests that part of the Eclipse platform may not be correctly initialized, but why doesn't it reproduce locally?

[eclipse-modisco-bot]

By Ed Willink on Nov 23, 2019 06:41

(In reply to Ed Willink from comment #10)

The StateObjectFactory create failure suggests that part of the Eclipse platform may not be correctly initialized, but why doesn't it reproduce locally?

Setting a breakpoint and running the JUnit tests locally shows that no attempt to create a IMPL_NAME = "org.eclipse.osgi.internal.resolver.StateObjectFactoryImpl" occurs.

org.eclipse.osgi.internal.resolver.StateObjectFactoryImpl is in the org.eclipse.osgi.compatibility.state whereas the invoking org.eclipse.osgi.service.resolver.StateObjectFactory$StateObjectFactoryProxy is in org.eclipse.osgi.

This seems familiar; the org.eclipse.osgi.compatibility.state fragment is missing despite using the lines from pom.xml copied from QVTd.

[eclipse-modisco-bot]

By Ed Willink on Nov 23, 2019 07:41

(In reply to Ed Willink from comment #11)

This seems familiar; the org.eclipse.osgi.compatibility.state fragment is missing despite using the lines from pom.xml copied from QVTd.

Using the 'Effective POM' tab reveals that the Modiso test POMs replace rather than augment te releng POM. Consequently org.eclipse.osgi.compatibility.state really is missing.

(In reply to Ed Willink from comment #10)

but why doesn't it reproduce locally?

Once org.eclipse.osgi.compatibility.state is present we may see an insightful error message.

[eclipse-modisco-bot]

By Ed Willink on Nov 23, 2019 10:01

(In reply to Ed Willink from comment #12)

(In reply to Ed Willink from comment #10)

but why doesn't it reproduce locally?

Once org.eclipse.osgi.compatibility.state is present we may see an insightful error message.

The stack traces do reproduce locally. It just seems to be the fail that varies.

[eclipse-modisco-bot]

By Ed Willink on Nov 23, 2019 10:52

(In reply to Ed Willink from comment #12)

Once org.eclipse.osgi.compatibility.state is present we may see an insightful error message.

Donw to just 10 fails; 8 marker prblme, 1 missing Acceleo file, 1 unexpected high memory (10% high)

All except the memory reproduce locally, where ther is no aggregator to show the errors distributed amongst 19 report files.

? the extra memory might be due to 128-bit rather than 64-bit memory allocations.

[eclipse-modisco-bot]

By Ed Willink on Nov 25, 2019 09:00

The initial push to master has 133 passes and 22 skips.

8 of the skips are due to failure to configure a target platform. Should be easy.

The final 1.2.0 build had 179 passes and 20 skips.

About 3 test modules were commented out during early Tycho struggles. It should be possible to get many of them working.

Bug remains open.

[eclipse-modisco-bot]

By Fabien Giquel on Nov 25, 2019 10:50

(In reply to Ed Willink from comment #14)

Donw to just 10 fails; 8 marker prblme, 1 missing Acceleo file, 1 unexpected high memory (10% high)

Hi Ed,\ about missing Acceleo file :

I have seen that you have fixed the java/jsp generations plugins in targeting at runtime directly the “frozen *.emtl files” contained in /emtl subfolder.\ Accordingly I have pushed to gerrit basic jsp.generation.tests additional fixes :\

• remove obsolete failing testEmtlFilePresent()\
• create missing AllTestsInUIThread.java to have test launched from maven configuration

Regards,

[eclipse-modisco-bot]

Nov 25, 2019 14:12

New Gerrit change created: https://git.eclipse.org/r/153331

[eclipse-modisco-bot]

Nov 25, 2019 16:47

New Gerrit change created: https://git.eclipse.org/r/153344

[eclipse-modisco-bot]

Nov 25, 2019 17:23

New Gerrit change created: https://git.eclipse.org/r/153348

[eclipse-modisco-bot]

Nov 25, 2019 17:47

Gerrit change https://git.eclipse.org/r/153348 was merged to [master].\ Commit: http://git.eclipse.org/c/modisco/org.eclipse.modisco.git/commit/?id=f7e2dd76a9546976cc1fb9288c07a62987b4618d

[eclipse-modisco-bot]

Nov 25, 2019 17:47

Gerrit change https://git.eclipse.org/r/153344 was merged to [master].\ Commit: http://git.eclipse.org/c/modisco/org.eclipse.modisco.git/commit/?id=7f9310e5a77ea70f7a01ec270148bae609fd35e8

[eclipse-modisco-bot]

By Ed Willink on Nov 25, 2019 17:51

(In reply to Ed Willink from comment #15)

The final 1.2.0 build had 179 passes and 20 skips.

Progress. All 199 tests now run, but with 37 skips.

[eclipse-modisco-bot]

By Ed Willink on Jan 11, 2020 13:53

(In reply to Ed Willink from comment #22)

Progress. All 199 tests now run, but with 37 skips.

Fixing Bug 559016 fixes UML initialization and supports model comparison.

18 test skips can beremoved. Unfortunately one new one is required. 20 skips.

[eclipse-modisco-bot]

By Ed Willink on Jan 13, 2020 10:38

Bug 559115 identifies serious UML2 currency issues requiring the org.eclipse.modisco.usecase.modelfilter.tests to be excluded from pom.xml.

The eventual 1.5.1M1 tally is: 196 tests 26 skipped.

[eclipse-modisco-bot]

By Ed Willink on Jan 16, 2020 11:34

Some of the intermittent failures are due to the same string being re-used as a catalog name. Test works fine alone, but incombination a major rename causes a conflict. A pity that the ElementWithSameNameException doesn't get thrown usefully.

Debug clue:

URI uri = catalog.getURI(this.name);

returns a renamed path with a disambiguating " (1)"