Bad Platform artifacts show up in build

[cdietrich]

in the builds with Java 8

https://github.com/itemis/xtext-reference-projects/runs/2835856963?check_suite_focus=true https://github.com/itemis/xtext-reference-projects/runs/2835856620?check_suite_focus=true

new plaform artifacts seem to be pulled this leads to build failing

 * What went wrong:
Execution failed for task ':org.xtext.example.mydsl:generateXtext'.
> com.google.common.util.concurrent.ExecutionError: com.google.common.util.concurrent.ExecutionError: com.google.common.util.concurrent.ExecutionError: com.google.common.util.concurrent.ExecutionError: com.google.common.util.concurrent.ExecutionError: java.lang.UnsupportedClassVersionError: org/eclipse/core/runtime/OperationCanceledException has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

2021-06-16T04:48:23.8460022Z 1    ERROR Mwe2Launcher       - org/eclipse/core/runtime/OperationCanceledException has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
2021-06-16T04:48:23.8465375Z java.lang.UnsupportedClassVersionError: org/eclipse/core/runtime/OperationCanceledException has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
2021-06-16T04:48:23.8469044Z    at java.lang.ClassLoader.defineClass1(Native Method)
2021-06-16T04:48:23.8471712Z    at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
2021-06-16T04:48:23.8474734Z    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
2021-06-16T04:48:23.8477789Z    at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
2021-06-16T04:48:23.8480558Z    at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
2021-06-16T04:48:23.8483125Z    at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
2021-06-16T04:48:23.8486274Z    at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
2021-06-16T04:48:23.8489455Z    at java.security.AccessController.doPrivileged(Native Method)
2021-06-16T04:48:23.8492711Z    at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
2021-06-16T04:48:23.8495707Z    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
2021-06-16T04:48:23.8508810Z    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
2021-06-16T04:48:23.8513500Z    at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.run(Mwe2Launcher.java:74)
2021-06-16T04:48:23.8518115Z    at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main(Mwe2Launcher.java:37)
2021-06-16T04:48:23.8523737Z    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:254)
2021-06-16T04:48:23.8527488Z    at java.lang.Thread.run(Thread.java:748)

[cdietrich]

[cdietrich]

emf ranges of death lead to new stuff pulled i simply dont understand the bom crap.

[LorenzoBettini]

Are the Xtext POM/BOM using version ranges or is it someone else? I always see lots of EMF and Eclipse POMs inspected when running the exec-maven-plugin

[cdietrich]

no the ranges are in emf. and the bom does not seem to be used for platform dependencies. and the optional gradle dependencies dont go into the pom

[cdietrich]

potential workaround for maven

<dependency>
    <groupId>org.eclipse.xtext</groupId>
    <artifactId>org.eclipse.xtext.xtext.generator</artifactId>
    <version>${xtextVersion}</version>
        <exclusions>
            <exclusion>
                <groupId>org.eclipse.platform</groupId>
                <artifactId>org.eclipse.equinox.common</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.eclipse.platform</groupId>
                <artifactId>org.eclipse.core.runtime</artifactId>
            </exclusion>
        </exclusions>
</dependency>

[LorenzoBettini]

the BOM is not used when running a Maven plugin: the BOM is used only when declaring project dependencies. If I understand correctly the problem shows up when running exec-maven-plugin right?

[cdietrich]

@LorenzoBettini yes its with the exec maven plugin. for gradle i have not found out yet

[LorenzoBettini]

Which is always a source of troubles :'( As I said, when you specify dependencies in any Maven plugin configuration/execution the BOM and other dependency management stuff are not used...

[cdietrich]

this also seems to break all our builds https://ci.eclipse.org/xtext/job/xtext-lib/job/master/46/console

[dhuebner]

Could we just run the build with Java11?

[cdietrich]

yes this is also a valid workaround (unfortunately this might lead to accidential use of java 11 api)

[oehme]

Coming here from the other issue @cdietrich created: I'm afraid I'm missing some context. Did Xtext upgrade its EMF version? Or does EMF have open version ranges that just pull in the latest stuff?

[cdietrich]

@oehme no Xtext uses emf, emf has ranges for core.runtime and equinox.common Xtext also has deps to core.runtime and equinox.common but some of them are optional and are not in pom so instead of the plaform version Xtext likes to have the newest are pulled

[oehme]

I just saw you have a BOM for this, which you already use here:

https://github.com/eclipse/xtext-core/blob/0a6606ee5091c48a716f74932bb6dd693cbee227/build.gradle#L32

You can add this BOM to the xtextTooling configuration (after the line that applies the Xtend plugin), then it will be used when resolving the dependencies of the incremental builder.

I.e.

dependencies {
    compile platform("org.eclipse.xtext:xtext-dev-bom:$project.version")    
}
if (findProperty('compileXtend') == 'true') {
    apply plugin: 'org.xtext.xtend'
    dependencies {
        xtextTooling platform("org.eclipse.xtext:xtext-dev-bom:$project.version")   
    }
}

[cdietrich]

this workaround seems to help as well

    configurations.all {
        resolutionStrategy { 
            eachDependency { DependencyResolveDetails details ->
                if (details.requested.group == 'org.eclipse.platform' && details.requested.name == 'org.eclipse.core.runtime') {
                    details.useVersion "3.19.0"
                }
                if (details.requested.group == 'org.eclipse.platform' && details.requested.name == 'org.eclipse.equinox.common') {
                    details.useVersion("3.13.0")
                }
            }
        }
    }

or

    configurations.all {
        if (name.endsWith("ToolingMain") || name.endsWith("ToolingTest")) {
            resolutionStrategy { 
                eachDependency { DependencyResolveDetails details ->
                    if (details.requested.group == 'org.eclipse.platform' && details.requested.name == 'org.eclipse.core.runtime') {
                        details.useVersion "3.19.0"
                    }
                    if (details.requested.group == 'org.eclipse.platform' && details.requested.name == 'org.eclipse.equinox.common') {
                        details.useVersion("3.13.0")
                    }
                }
            }
        }
    }

[cdietrich]

You can add this BOM to the xtextTooling configuration (after the line that applies the Xtend plugin), then it will be used when resolving the dependencies of the incremental builder.

@oehme any hint on how to do that

[oehme]

dependencies {
    compile platform("org.eclipse.xtext:xtext-dev-bom:$project.version")    
}
if (findProperty('compileXtend') == 'true') {
    apply plugin: 'org.xtext.xtend'
    dependencies {
        xtextTooling platform("org.eclipse.xtext:xtext-dev-bom:$project.version")   
    }
}

[cdietrich]

tried this but it does not seems to work

> Could not find method xtextTooling() for arguments [DefaultExternalModuleDependency{group='org.eclipse.xtext', name='xtext-dev-bom', version='2.26.0-SNAPSHOT', configuration='default'}] on object of type org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler.
same with xtendTooling
> Could not find method xtendTooling() for arguments [DefaultExternalModuleDependency{group='org.eclipse.xtext', name='xtext-dev-bom', version='2.26.0-SNAPSHOT', configuration='default'}] on object of type org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler.

[oehme]

Ah right, that's because there's one for each sourceSet. In that case we need loop:

configurations.all {
    if (name.endsWith("Tooling")) {
        dependencies {
            add(name, platform("org.eclipse.xtext:xtext-dev-bom:$project.version"))
        }
    }
}

[cdietrich]

unfortunately this still does not work. it now fails to find the XtendStandaloneSetup class also the configurations seems to be ToolingTest and ToolingMain

[cdietrich]

the transitive hell is also in platform artifacts

[cdietrich]

it seems to work to manage this will exclusions manually, but how to find all places and exclustions https://github.com/eclipse/xtext-core/compare/cd_xtext_issue1976 https://github.com/eclipse/xtext-xtend/compare/cd_xtext_issue1976

[oehme]

unfortunately this still does not work. it now fails to find the XtendStandaloneSetup class

Ah damn it, that's because I used defaultDependencies in the Xtend plugin (which I shouldn't have done). So please adjust the workaround to:

configurations.all {
    if (name.contains("Tooling")) {
        dependencies {
            add(name, "org.eclipse.xtend:org.eclipse.xtend.core:$project.version"))
            add(name, platform("org.eclipse.xtext:xtext-dev-bom:$project.version"))
        }
    }
}

[oehme]

it seems to work to manage this will exclusions manually, but how to find all places and exclustions

You shouldn't need any exclusions - The fix above (applying the Xtext BOM) is the way to go. I can make applying that BOM easier in a future version, but for now the code above should do.

[cdietrich]

yes, but it does not seem to work yet. https://ci.eclipse.org/xtext/job/xtext-lib/job/cd_xtext_issue1976b/2/console https://github.com/eclipse/xtext-lib/compare/cd_xtext_issue1976b the solution also needs to work with pure maven

[oehme]

Can you enable build scans, so I can easily see the dependencies that were resolved during those CI builds?

the solution also needs to work with pure maven

So do I understand correctly: Xtext depends on a newer version of EMF. EMF pulls in a platform version that no longer runs on Java 8. But Xtext still wants to run on Java 8. In that case I'd say the EMF upgrade needs to be reverted. Please correct me if I understood the situation wrong.

[cdietrich]

no, xtext still depends on the same emf.

but emf (and also jdt have other platform dependencies in the style [3.12.0,4.0.0) xtext bom pins them to a wanted/tested version e.g 3.13.0 now platform and jdt publish new maven artifacts e.g. 3.15.0 which is now resolved instead of the wanted 3.13.0

[cdietrich]

build scan can be found here https://scans.gradle.com/s/auzttsmqbodtm we seem to have a xtendCompiler config too

[oehme]

build scan can be found here https://scans.gradle.com/s/auzttsmqbodtm

Thanks, I can now see that we're not using the xxxTooling configurations at all, but we set a custom Xtend classpath. I had forgotten about this. It's in this build script. So you need to add the BOM there. Yay for build scans :)

[cdietrich]

still does not seem to work, but now gradle scans is down https://ci.eclipse.org/xtext/job/xtext-lib/job/cd_xtext_issue1976b/4/console

[oehme]

Okay, I'll check out the project and get back to you.

[cdietrich]

@oehme i am just stupid. deleted the platform method accidentially, put chaning seems not be enhough

[oehme]

I had a look and the problem is that the xtext-dev-bom does not mark these as "strict" versions and thus Gradle merely treats them as minimum requirements. I.e. this line should probably be:

api("org.eclipse.platform:org.eclipse.equinox.common") { version { strictly("3.13.0") } }

This applies to any version where you can't deal with upgrades. The Gradle user guide has a nice summary of all the ways you can express your version constraints, so you can see which applies best to each situation.

[cdietrich]

@oehme i assume we then need to do this for all lines?!?

i also dont see where this information is preserved in maven as we do have the code split on 8 repos where 6 or so build with gradle the dev bom is consumed from upstream just as maven artifact

      <dependency>
        <groupId>org.eclipse.platform</groupId>
        <artifactId>org.eclipse.equinox.common</artifactId>
        <version>3.13.0</version>
        <scope>compile</scope>
      </dependency>

[oehme]

Correcting myself: The BOM is already enough, the problem was that Gradle wasn't using it, because BOM support is specific to the java plugin and the root project did not have that plugin, so it just ignored the contents of the BOM. See https://github.com/eclipse/xtext-lib/pull/386 for the solution.

[cdietrich]

cool. now i need to figure out if the same can somehow be applied to https://github.com/itemis/xtext-reference-projects/tree/master/greetings-gradle/2.26.0

[oehme]

Now for the xtext-gradle-plugin, I'm planning a new release anyway. For that I'll add the BOM automatically, so normal Xtext users won't need any additional code. If you want to fix the reference projects in the meantime, you can do the workaround I had described earlier:

configurations.all {
    if (name.contains("Tooling")) {
        dependencies {
            add(name, "org.eclipse.xtend:org.eclipse.xtend.core:$xtextVersion"))
            add(name, platform("org.eclipse.xtext:xtext-dev-bom:$xtextVersion"))
        }
    }
}

The Maven plugin should import the BOM in its own POM.

[cdietrich]

hmmm the xtext-maven plugin and the xtend-maven-plugin already make use of the bom (via their parents) https://github.com/eclipse/xtext-maven/blob/a771b4b8b9589d07391931f362332b2710697a1b/org.eclipse.xtext.maven.parent/pom.xml#L27 https://github.com/eclipse/xtext-xtend/blob/ae80a5c1bb09a61596cb9848331694d8ebc562ab/releng/org.eclipse.xtend.maven.parent/pom.xml#L33

thus the problem should affect the maven-exec-plugin sections only

[oehme]

Ah I didn't know it was only about the exec plugin. That's odd though, since it uses the classpath of your project, which is managed by the BOM.

[cdietrich]

we have an explicit dependency list for it https://github.com/itemis/xtext-reference-projects/blob/f0f5fdd938dc352a0a5d42a1d6eece5e63377890/greetings-tycho/2.26.0/org.xtext.example.mydsl/pom.xml#L40

[oehme]

Ah I see, that's problematic since you can't put a BOM there. I'm afraid you'll have to explicitly list the desired versions or exclude the undesired transitives.

[LorenzoBettini]

Yes, that's what I said in https://github.com/eclipse/xtext/issues/1976#issuecomment-862096854 I was wondering if we can get rid of <includePluginDependencies>true</includePluginDependencies> so that we could simply specify org.eclipse.emf.mwe2.launch as a dependency of the exec plugin configuration and take the other deps from the POM of the project? Who was the first to introduce the exec plugin configuration?

[oehme]

Isn't mwe2.launch the one that pulls in the problematic deps?

[LorenzoBettini]

By looking at https://github.com/eclipse/xtext/issues/1976#issuecomment-862095532 I thought it was xtext.generator

[cdietrich]

the problem is not the launch, it is the maven exec. the launch config uses the target platform, so if you use newer eclipse you will have to use java 11 anyway. so this is no issue

[ewillink]

See https://bugs.eclipse.org/bugs/show_bug.cgi?id=574259

The change to BREE 11 for org.eclipse.core makes anything non-trivial using Java 8 a thing of the past.

[LorenzoBettini]

But I seem to understand the problem is different here: you want to build with Java 8 with an older version of Eclipse. With the BOM you have full control on dependencies for the project's dependencies but not for the exec plugin configuration dependencies (you have to manually set exclusions for unwanted dependencies in the exec plugin configuration, that's why I was wondering whether we can have the exec plugin configuration to reuse the project's dependencies, https://www.mojohaus.org/exec-maven-plugin/java-mojo.html#includePluginDependencies)

[cdietrich]

@LorenzoBettini i have the feeling we were doing this in the past, but also changed that. https://github.com/itemis/xtext-reference-projects/blob/bd7ff370f35e8aeee191e1c68834b05451f30538/greetings-maven/2.13.0/org.xtext.example.mydsl/pom.xml#L18

i have not looked up why

[LorenzoBettini]

By the way, this problem only shows up with Gradle right? I cannot reproduce that with Maven locally: I created a new Xtext project with Maven support (using Xtext 2.25) and Java 8, then changed the Xtext version in the parent POM with 2.26.0-SNAPSHOT and enabled Sonatype snapshot repository. The build runs fine with Java 8 (also the exec plugin).

[cdietrich]

@LorenzoBettini in maven it happens with tycho projects only https://github.com/itemis/xtext-reference-projects/runs/2835856963?check_suite_focus=true