Closed cdietrich closed 10 months ago
I am working on this as well, or plan too do that in the coming days.
Locally I have the build producing "proper update sites" like this:
This is like what I've done for a bunch of other projects...
Hopefully you guys are into that. I don't want to waste my time...
Help is welcome
Very good!
I believe I can prototype everything from a fork, including prototyping the build from my fork, because I can use one of my other ci instance to build my fork directly. That way I'll have a fully functional proof of concept when I create a pull request later on...
I'm pleased to report good progress.
This prototype build:
https://ci.eclipse.org/oomph/job/internal/job/xpect/
produces this update site:
https://download.eclipse.org/oomph/archive/xpect/updates/nightly/latest
The changes are on the master branch of my fork:
https://github.com/merks/Xpect
This is the declarative pipeline:
@cdietrich
One huge request I have here is that Xpect produces a release compatible with Xtext's 2023-09 release before you start doing the migration to the new bundle symbolic names. Please please!
@merks i dont understand what you are referring to.
I am working on that and I will help with that. It's pretty much a trivial activity with the promotion infrastructure I'm adding right now...
I just noticed that building against the nightly build target doesn't actually work because of the classgraph bundle name change so given I have a bunch of changes in the works, I'm hoping that you are not making changes to xpect master while my changes are pending...
We can hold changes back until you are finished with your contribution
Thank you. đ
Will you be at EclipseCon next week so we can chat in person?
The prototype is pretty much complete. This multi-branch pipeline does the builds:
https://ci.eclipse.org/oomph/job/internal/job/xpect/
The builds are parameterized with these as the default parameter values:
I fixed the nightly build target platform to work as well:
Doing milestone and release builds is just a matter of selecting that in the drop-down:
Here is the resulting update site after doing a milestone and a final release build:
https://download.eclipse.org/oomph/archive/xpect/updates/release/latest/
Note that the URLs displayed are for where the actual update sites will be when Xpect produces them.
@cdietrich
I don't have much time left to create a PR and tomorrow I travel to the conference...
All the changes are in this commit:
https://github.com/merks/Xpect/commit/master
If I were a committer, I could do all the final tweaking myself...
@merks many thanks. the pr looks very nice. unfortunately i wont make it to eclipsecon
đ I was really really hoping to see in person again!
new job still keeps me very busy.
Well, that was one of the things I wanted to hear about!!
done
@cdietrich
Look, it published the repo!
https://download.eclipse.org/xpect/updates/nightly/latest
I tested the real setup and it worked too.
https://github.com/eclipse/Xpect/blob/master/CONTRIBUTING.md
Could I ask you to kick off the build manually and choose "milestone"? So that we have our first milestone?
Do you want me to change the pipeline so that there is no target platform choice but rather both are built? The nightly one would be used only for test verification and promotion would only apply to the other one. Okay?
nice. @merks please do so, but take your time.
@merks , how do I build Xpect locally, with your change? I get asked for a password:
[INFO] --- tycho-eclipserun:3.0.0:eclipse-run (promote) @ org.eclipse.xpect.repository.promotion ---
[INFO] Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/tycho/extras/tycho-p2-extras-plugin/3.0.0/tycho-p2-extras-plugin-3.0.0.pom
[INFO] Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/tycho/extras/tycho-p2-extras-plugin/3.0.0/tycho-p2-extras-plugin-3.0.0.pom (5.5 kB at 500 kB/s)
[INFO] Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/tycho/extras/tycho-p2-extras-plugin/3.0.0/tycho-p2-extras-plugin-3.0.0.jar
[INFO] Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/tycho/extras/tycho-p2-extras-plugin/3.0.0/tycho-p2-extras-plugin-3.0.0.jar (31 kB at 3.5 MB/s)
[INFO] ### Using TychoRepositoryTransport for remote P2 access ###
[INFO] Cache location: /data/git/Xpect/xpect-local-maven-repository/.cache/tycho
[INFO] Transport mode: online
[INFO] Update mode: forced
[INFO] Minimum cache duration: 60 minutes
[INFO] (you can configure this with -Dtycho.p2.transport.min-cache-minutes=<desired minimum cache duration>)
[INFO] Adding repository https://download.eclipse.org/releases/2023-09
[INFO] Adding repository https://download.eclipse.org/justj/tools/updates/nightly/latest
[INFO] Expected eclipse log file: /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/eclipserun-work/data/.metadata/.log
[INFO] Command line:
[/usr/lib/jvm/jdk-17_adoptium/bin/java, -jar, /data/git/Xpect/xpect-local-maven-repository/p2/osgi/bundle/org.eclipse.equinox.launcher/1.6.500.v20230717-2134/org.eclipse.equinox.launcher-1.6.500.v20230717-2134.jar, -install, /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/eclipserun-work, -configuration, /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/eclipserun-work/configuration, -data, /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/eclipserun-work/data, -consoleLog, -application, org.eclipse.justj.p2.manager, -data, @None, -nosplash, -remote, genie.xpect@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/xpect, -retain, 5, -label, Xpect, -build-url, http://www.example.com/, -root, /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/xpect-sync, -relative, updates, -excluded-categories-pattern, .*.Default, -iu-filter-pattern, org.eclipse.xpect.*, -version-iu, org.eclipse.xpect.sdk., -commit, https://github.com/eclipse/Xpect/commit/${git.commit}, -target-url, https://download.eclipse.org/xpect, -promote, /data/git/Xpect/org.eclipse.xpect.releng/promotion/../p2-repository/target/repository, -timestamp, 2023-10-20T15:24:34Z, -type, nightly, -breadcrumb, Xpect https://projects.eclipse.org/projects/modeling.xpect, -favicon, https://eclipseide.org/favicon.ico]
!SESSION 2023-10-20 17:29:09.826 -----------------------------------------------
eclipse.buildId=unknown
java.version=17.0.6
java.vendor=Eclipse Adoptium
BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US
Framework arguments: -application org.eclipse.justj.p2.manager -remote genie.xpect@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/xpect -retain 5 -label Xpect -build-url http://www.example.com/ -root /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/xpect-sync -relative updates -excluded-categories-pattern .*.Default -iu-filter-pattern org.eclipse.xpect.* -version-iu org.eclipse.xpect.sdk. -commit https://github.com/eclipse/Xpect/commit/${git.commit} -target-url https://download.eclipse.org/xpect -promote /data/git/Xpect/org.eclipse.xpect.releng/promotion/../p2-repository/target/repository -timestamp 2023-10-20T15:24:34Z -type nightly -breadcrumb Xpect https://projects.eclipse.org/projects/modeling.xpect -favicon https://eclipseide.org/favicon.ico
Command-line arguments: -data /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/eclipserun-work/data -consoleLog -application org.eclipse.justj.p2.manager -data @None -remote genie.xpect@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/xpect -retain 5 -label Xpect -build-url http://www.example.com/ -root /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/xpect-sync -relative updates -excluded-categories-pattern .*.Default -iu-filter-pattern org.eclipse.xpect.* -version-iu org.eclipse.xpect.sdk. -commit https://github.com/eclipse/Xpect/commit/${git.commit} -target-url https://download.eclipse.org/xpect -promote /data/git/Xpect/org.eclipse.xpect.releng/promotion/../p2-repository/target/repository -timestamp 2023-10-20T15:24:34Z -type nightly -breadcrumb Xpect https://projects.eclipse.org/projects/modeling.xpect -favicon https://eclipseide.org/favicon.ico
!ENTRY org.eclipse.justj.p2 2 0 2023-10-20 17:29:11.933
!MESSAGE Starting the p2 manager
/usr/bin/ssh genie.xpect@projects-storage.eclipse.org mkdir -p /home/data/httpd/download.eclipse.org/xpect/updates
genie.xpect@projects-storage.eclipse.org's password:
Simply disable the profile does not work ? -P!promote
Note too that there are these run configurations:
They should all work locally. Even promotion can be tested locally:
@merks I can't build Xpect locally anymore, I get asked for a password:
[INFO] --- tycho-eclipserun:4.0.3:eclipse-run (promote) @ org.eclipse.xpect.repository.promotion ---
[INFO] ### Using TychoRepositoryTransport for remote P2 access ###
[INFO] Cache location: /data/git/Xpect/xpect-local-maven-repository/.cache/tycho
[INFO] Transport mode: online
[INFO] Http Transport type: Java11Client
[INFO] Update mode: forced
[INFO] Minimum cache duration: 60 minutes
[INFO] (you can configure this with -Dtycho.p2.transport.min-cache-minutes=<desired minimum cache duration>)
[INFO] Adding repository https://download.eclipse.org/releases/2023-09
[INFO] Adding repository https://download.eclipse.org/justj/tools/updates/nightly/latest
[INFO] Expected Eclipse log file: /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/eclipserun-work/data/.metadata/.log
[INFO] Command line:
[/usr/lib/jvm/jdk-17_adoptium/bin/java, -jar, /data/git/Xpect/xpect-local-maven-repository/p2/osgi/bundle/org.eclipse.equinox.launcher/1.6.500.v20230717-2134/org.eclipse.equinox.launcher-1.6.500.v20230717-2134.jar, -install, /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/eclipserun-work, -configuration, /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/eclipserun-work/configuration, -data, /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/eclipserun-work/data, -consoleLog, -application, org.eclipse.justj.p2.manager, -data, @None, -nosplash, -remote, genie.xpect@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/xpect, -retain, 5, -label, Xpect, -build-url, http://www.example.com/, -root, /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/xpect-sync, -relative, updates, -excluded-categories-pattern, .*.Default, -iu-filter-pattern, org.eclipse.xpect.*, -version-iu, org.eclipse.xpect.sdk., -commit, https://github.com/eclipse/Xpect/commit/${git.commit}, -target-url, https://download.eclipse.org/xpect, -promote, /data/git/Xpect/org.eclipse.xpect.releng/promotion/../p2-repository/target/repository, -timestamp, 2023-10-20T14:54:38Z, -type, nightly, -breadcrumb, Xpect https://projects.eclipse.org/projects/modeling.xpect, -favicon, https://eclipseide.org/favicon.ico]
!SESSION 2023-10-20 16:57:45.584 -----------------------------------------------
eclipse.buildId=unknown
java.version=17.0.6
java.vendor=Eclipse Adoptium
BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US
Framework arguments: -application org.eclipse.justj.p2.manager -remote genie.xpect@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/xpect -retain 5 -label Xpect -build-url http://www.example.com/ -root /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/xpect-sync -relative updates -excluded-categories-pattern .*.Default -iu-filter-pattern org.eclipse.xpect.* -version-iu org.eclipse.xpect.sdk. -commit https://github.com/eclipse/Xpect/commit/${git.commit} -target-url https://download.eclipse.org/xpect -promote /data/git/Xpect/org.eclipse.xpect.releng/promotion/../p2-repository/target/repository -timestamp 2023-10-20T14:54:38Z -type nightly -breadcrumb Xpect https://projects.eclipse.org/projects/modeling.xpect -favicon https://eclipseide.org/favicon.ico
Command-line arguments: -data /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/eclipserun-work/data -consoleLog -application org.eclipse.justj.p2.manager -data @None -remote genie.xpect@projects-storage.eclipse.org:/home/data/httpd/download.eclipse.org/xpect -retain 5 -label Xpect -build-url http://www.example.com/ -root /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/xpect-sync -relative updates -excluded-categories-pattern .*.Default -iu-filter-pattern org.eclipse.xpect.* -version-iu org.eclipse.xpect.sdk. -commit https://github.com/eclipse/Xpect/commit/${git.commit} -target-url https://download.eclipse.org/xpect -promote /data/git/Xpect/org.eclipse.xpect.releng/promotion/../p2-repository/target/repository -timestamp 2023-10-20T14:54:38Z -type nightly -breadcrumb Xpect https://projects.eclipse.org/projects/modeling.xpect -favicon https://eclipseide.org/favicon.ico
!ENTRY org.eclipse.justj.p2 2 0 2023-10-20 16:57:47.564
!MESSAGE Starting the p2 manager
/usr/bin/ssh genie.xpect@projects-storage.eclipse.org mkdir -p /home/data/httpd/download.eclipse.org/xpect/updates
genie.xpect@projects-storage.eclipse.org's password:
a
a
genie.xpect@projects-storage.eclipse.org's password:
genie.xpect@projects-storage.eclipse.org's password:
Warning: Permanently added 'projects-storage.eclipse.org' (ECDSA) to the list of known hosts.
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,password).
exitValue=255
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for org.eclipse.xpect.root 0.3.0-SNAPSHOT:
[INFO]
[INFO] org.eclipse.xpect.parent ........................... SUCCESS [ 0.512 s]
[INFO] org.eclipse.xpect.mwe2 ............................. SUCCESS [01:02 min]
[INFO] org.eclipse.xpect .................................. SUCCESS [ 28.250 s]
[INFO] org.eclipse.xpect.ide .............................. SUCCESS [ 2.201 s]
[INFO] org.eclipse.xpect.ui ............................... SUCCESS [ 9.095 s]
[INFO] org.eclipse.xpect.ui.junit ......................... SUCCESS [ 5.303 s]
[INFO] org.eclipse.xpect.xtext.lib ........................ SUCCESS [ 2.002 s]
[INFO] org.eclipse.xpect.xtext.xbase.lib .................. SUCCESS [ 2.279 s]
[INFO] org.eclipse.xpect.xtext.lib.feature ................ SUCCESS [ 1.323 s]
[INFO] org.eclipse.xpect.xtext.xbase.lib.feature .......... SUCCESS [ 0.632 s]
[INFO] org.eclipse.xpect.sdk .............................. SUCCESS [ 0.858 s]
[INFO] org.eclipse.xpect.repository ....................... SUCCESS [ 4.014 s]
[INFO] org.eclipse.xpect.target.eclipse_2023_09-xtext_nightly SUCCESS [ 0.009 s]
[INFO] org.eclipse.xpect.root ............................. SUCCESS [ 0.017 s]
[INFO] org.eclipse.xpect.repository.promotion ............. FAILURE [01:41 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:34 min
[INFO] Finished at: 2023-10-20T16:59:12+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.eclipse.tycho.extras:tycho-eclipserun-plugin:4.0.3:eclipse-run (promote) on project org.eclipse.xpect.repository.promotion: Error while executing platform: Error while executing platform: return code=255, see content of /data/git/Xpect/org.eclipse.xpect.releng/promotion/target/eclipserun-work/data/.metadata/.logfor more details. -> [Help 1]
Should this promotion really be enabled by default?
I played with that and it somehow seems as if the profiles could not be independently configured. Not specifying the target explicitly also didnât work (for me). So there are checked in launchers that do the correct thing and theyâre easy to use.
Sorry for the double post, I didn't notice the 2nd time I was writing a comment, that I already wrote one.
Simply disable the profile does not work ? -P!promote
Yes, this works. But why do I have to add an extra argument just to make the build work?
I think I tried to answer that. When I made that one not active by default it disabled them all by default. Not what I expectedâŠ. Otherwise I would have done it like that.
I think I tried to answer that. When I made that one not active by default it disabled them all by default. Not what I expectedâŠ. Otherwise I would have done it like that.
Ah, sorry then, I didn't understand your comment.
Maybe https://github.com/eclipse/Xpect#readme can be updated to include -P!promote
then? That'd work for me.
Yes the readme needs updating. Also to document the new update sites. I plan to follow up on that next week.
we should convert the current scripted pipeline Jenkinsfile to a declarative one (analogous to Xtext and MWE) cc @tjeske