Move to Maven/Tycho for projects and build infrastructure

LorenzoBettini commented 1 year ago

See #499

Please see https://github.com/eclipse/xtext-core/pull/2054 for some details for testing this manually (of course, the git repo is this one).

More details on the relevant changes I made and what's left to do will follow shortly.

LorenzoBettini commented 1 year ago

Here are a few detailed notes:

created a new parent POM (the project xtext-lib)
xtext.dev-bom It has no parent, but the parent POM lists it as a module (so that it will be deployed to Maven central) and imports it so that it will be available to all the children. The alternative would be to make the dev-bom the parent project of xtext-lib.
xtend.lib.macro It's ODD that the groupId is different: org.eclipse.xtend instead of org.eclipse.xtext... maybe we should do another parent POM? Maybe after we ported also xtend?

xbase.lib.tests is not even an Eclipse plug-in project transformed in a plug-in project and added this to the TP:

    <location includeDependencyDepth="none" includeDependencyScopes="compile" includeSource="true" missingManifest="generate" type="Maven">
        <dependencies>
            <dependency>
                <groupId>com.google.guava</groupId>
                <artifactId>guava-testlib</artifactId>
                <version>30.1-jre</version>
                <type>jar</type>
            </dependency>
        </dependencies>
        <instructions><![CDATA[
Bundle-Name:           Bundle derived from maven artifact ${mvnGroupId}:${mvnArtifactId}:${mvnVersion}
version:               ${version_cleanup;${mvnVersion}}
Bundle-SymbolicName:   ${mvnGroupId}.${mvnArtifactId}
Bundle-Version:        ${version}
Import-Package:        *;resolution:=optional
Export-Package:        *;version="${version}";-noimport:=true
DynamicImport-Package: *
]]></instructions>
    </location>

because guava-testlib is not in Orbit. I don't know if Oomph will be able to handle that. The alternative is to turn it into a plain Maven project and take the dependency directly from Maven central. I can investigate.

concerning tests in this project: In Eclipse it runs 1031 tests. In Maven, enabling also $ classes (it's required otherwise by default maven-surefire ignores inner classes) we have Tests run: 1766. Thus, many more. That's because it executes MapExtensionsTest, MapExtensionsTest$ManualTest and MapExtensionsTest$GuavaTest. So it basically executes the same tests twice. That's harmless, tests are really fast!
xtend.lib.tests OK (only one test)
gwt, It's not in an update site. It's deployed only on Maven central https://central.sonatype.dev/artifact/org.eclipse.xtext/org.eclipse.xtext.xbase.lib.gwt/2.30.0.M1. Gwt projects have been updated successfully and the gwt test project has been converted to Maven as well. The gwt plugin is used to verify that gwt can compile it. The gwt projects are correctly processed with Maven instead of Gradle, to copy the sources filtering only the ones gwt compliant.
Old xbase.lib feature renamed to xbase.lib.feature: does not include the source of xbase.lib bundle, the source feature will be generated by Tycho and inserted in the p2 repository

added a new xtend.lib.feature

<includes
     id="org.eclipse.xtext.xbase.lib.feature"
     version="0.0.0"/>

<plugin
     id="org.eclipse.xtend.lib"
     download-size="0"
     install-size="0"
     version="0.0.0"
     unpack="false"/>

<plugin
     id="org.eclipse.xtend.lib.macro"
     download-size="0"
     install-size="0"
     version="0.0.0"
     unpack="false"/>

new project org.eclipse.xtext.lib.p2repository including only the two features above and their sources (I guess that's another temporary repository that will go away if we convert all the projects, and maybe merge the git repositories)
the old releng directory has been removed

cdietrich commented 1 year ago

@LorenzoBettini there seems to be something broken with the features. i rebased your changes onto lb_2052_maven_tycho branch in lib, and now https://ci.eclipse.org/xtext/job/xtext-eclipse/job/lb_2052_maven_tycho/6/console

fails due missing features

looking at https://ci.eclipse.org/xtext/job/xtext-lib/job/lb_2052_maven_tycho/lastSuccessfulBuild/artifact/build/p2-repository/features/ it might be a naming convention thing

LorenzoBettini commented 1 year ago

@LorenzoBettini there seems to be something broken with the features. i rebased your changes onto lb_2052_maven_tycho branch in lib, and now https://ci.eclipse.org/xtext/job/xtext-eclipse/job/lb_2052_maven_tycho/6/console

fails due missing features

looking at https://ci.eclipse.org/xtext/job/xtext-lib/job/lb_2052_maven_tycho/lastSuccessfulBuild/artifact/build/p2-repository/features/ it might be a naming convention thing

I guess so: I introduced a new xbase.lib.feature that probably is also the name of the real xbase.lib.feature. (previously, it was simply called xbase.lib, which would have conflicted with the xbase.lib project in the workspace).

If you want, I can rename that feature here in xtext-lib, also because it's meant to be temporary. Or I can simply do without features and create a create a (temporary) p2 repository with just bundles?

cdietrich commented 1 year ago

the feature with that name also seems to be here https://ci.eclipse.org/xtext/job/xtext-umbrella/job/master/lastSuccessfulBuild/artifact/build/p2-repository/features/ so i guess a rename would break compatibility is it possible to have a different name in .project and in the feature?

LorenzoBettini commented 1 year ago

I guess so, but as I said, I can also do without features as it was before in this project (and just include bundles in the p2 repository), what do you think?

cdietrich commented 1 year ago

i dont know who downstream uses the features. thus i wonder if there is a way to keep them. see also https://github.com/eclipse/xtext/tree/v2.10.0/features here the project name and feature id seem to differ from each other

is this not possible with tycho?

cc @szarnekow

LorenzoBettini commented 1 year ago

@cdietrich I'll give it a try!

LorenzoBettini commented 1 year ago

The project name is not important, but the artifactId and the feature id must be the same:

[ERROR] Failed to execute goal org.eclipse.tycho:tycho-packaging-plugin:2.7.5:validate-id (default-validate-id) on project org.eclipse.xtext.xbase.lib.feature: The Maven artifactId (currently: "org.eclipse.xtext.xbase.lib.feature") must be the same as the feature ID (currently: "org.eclipse.xtext.xbase.lib") -> [Help 1]

And if I change the artifactId according to the feature name two artifacts with the same name are detected:

[ERROR] [ERROR] Project 'org.eclipse.xtext:org.eclipse.xtext.xbase.lib:2.30.0-SNAPSHOT' is duplicated in the reactor @

What I can do is try avoiding the features completely and see if the downstream jobs fail.

It's Maven that prohibits that, I'd say.

cdietrich commented 1 year ago

@laeubi do you know anything to bypass this?

laeubi commented 1 year ago

Having two artifacts with the same Group/Artifact ID is not allowed (and most likely will sooner or later be a source of confusion), so often the convention is to name features <id>.feature and plugins <id> so there is no clash.

Nerveless you can disable the check for the ID by explicitly configure the default-validate-id execution, but do this on your own risk, there is likely no support from Tycho for such configurations and probably just a few using that for very special use-cases.

LorenzoBettini commented 1 year ago

@LorenzoBettini there seems to be something broken with the features. i rebased your changes onto lb_2052_maven_tycho branch in lib, and now https://ci.eclipse.org/xtext/job/xtext-eclipse/job/lb_2052_maven_tycho/6/console fails due missing features looking at https://ci.eclipse.org/xtext/job/xtext-lib/job/lb_2052_maven_tycho/lastSuccessfulBuild/artifact/build/p2-repository/features/ it might be a naming convention thing

I guess so: I introduced a new xbase.lib.feature that probably is also the name of the real xbase.lib.feature. (previously, it was simply called xbase.lib, which would have conflicted with the xbase.lib project in the workspace).

If you want, I can rename that feature here in xtext-lib, also because it's meant to be temporary. Or I can simply do without features and create a create a (temporary) p2 repository with just bundles?

@cdietrich Having a look again at this message of yours, you refer to

https://ci.eclipse.org/xtext/job/xtext-lib/job/lb_2052_maven_tycho/lastSuccessfulBuild/artifact/build/p2-repository/features/

but that's a wrong job...

this is the right one

https://ci.eclipse.org/xtext/job/xtext-lib/job/lb_499_maven_tycho/

I don't understand what went wrong... maybe I pushed a wrong branch (I'll remove it).

Could you please modify the xtext-eclipse job so that it uses the right Jenkins job, where I removed the features from the p2 repository? Sorry for the confusion.

cdietrich commented 1 year ago

@LorenzoBettini the problem is you used a different branch name foreach repo so i took your changes and merged it to the other branch so you have to update the branch i had created

LorenzoBettini commented 1 year ago

@LorenzoBettini the problem is you used a different branch name foreach repo so i took your changes and merged it to the other branch so you have to update the branch i had created

I've just removed that branch from this GitHub repository. I've used a branch name with the issue number of this very repository. Also the PR is based on my branch. So what should I do? Shall we keep the branch you had created? But then you have to recreate it. And what about the PR?

cdietrich commented 1 year ago

i will merge your changes to my branch

cdietrich commented 1 year ago

the problem is: the build downstream branch wants to use the same branch name along the line of course you can also change all downstream branches manually but this pita pushed the branch in new version again

LorenzoBettini commented 1 year ago

OK, so let's use your branch and I'll update the PR

LorenzoBettini commented 1 year ago

No, the PR cannot be modified in that respect... I'll close this and create another one, OK?

cdietrich commented 1 year ago

thats ok

LorenzoBettini commented 1 year ago

Superseded by #501

eclipse / xtext-lib

Move to Maven/Tycho for projects and build infrastructure #500