Please stage p2 repository too

[akurtakov]

Consumers relying on p2 have to wait till the release is done before testing or go through complicated build magics to handcraft the repo. It would be really nice if staged p2 repo is also generated so projects like Eclipse platform can verify the version is good before released.

[jmcc0nn3ll]

I have a hard time seeing us doing this, the creation of a p2 repository from my perspective is strictly an after release 'extra' that is more a holdover from our time on the release train then anything else. It would probably be safer and better if someone who actually used p2 created and maintained it, none of us use them. If someone is relying on them is relying on them in a production capacity it would be good to know.

[akurtakov]

Can you point to how you generate the p2 repo and what the staging process is? Maybe we can find someone spending the time to hook the former into the later or to simplify it so it's a no-brainer to be run.

[jmcc0nn3ll]

http://git.eclipse.org/c/jetty/org.eclipse.jetty.releng.bundles.git

That is where we have the bits for generating the p2 repository. We publish it through a build at:

https://ci.eclipse.org/shared/view/Jetty-RT/job/jetty-rt-bundles-9

We do it after the artifacts are in maven central because the process just pulls them from central and does that hinky pack, sign, unpack, pack, whatever thing that was required on the release train. It then puts the finished product in the download.eclipse.org/jetty p2 location. We could have another build that pulls from a snapshot repository, does the funky signing goop and publishes to another snapshotish p2 repository but not something I think we want to chase down.

If you have a better way to do all of this I am all ears, this entire p2 process has been causing me heartburn for years.

[joakime]

@akurtakov any updated thoughts on taking the p2 repository role over?

[rgrunber]

It should be possible to have the pack/sign logic as part of the individual module builds on Eclipse infrastructure. At Orbit we do it as follows http://git.eclipse.org/c/gerrit/orbit/orbit-recipes.git/tree/releng/mavenparent/pom.xml#n176 (using tycho-pack200a-plugin, eclispe-jarsigner-plugin, tycho-pack200b-plugin)

There's also articles like http://www.codetrails.com/blog/sign-your-eclipse-project that explain how to do it.

[jmcc0nn3ll]

Ideally someone with interest in p2 repositories should take over this process, we have a signing process for Maven artifacts but do not build actual binaries on any of the eclispe build machines.

On Jun 19, 2017 13:48, "Roland Grunberg" notifications@github.com wrote:

It should be possible to have the pack/sign logic as part of the build process on Eclipse infrastructure. At Orbit we do it as follows http://git.eclipse.org/c/gerrit/orbit/orbit-recipes. git/tree/releng/mavenparent/pom.xml#n176 (using tycho-pack200a-plugin, eclispe-jarsigner-plugin, tycho-pack200b-plugin)

There's also articles like http://www.codetrails.com/ blog/sign-your-eclipse-project that explain how to do it.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/eclipse/jetty.project/issues/1587#issuecomment-309531197, or mute the thread https://github.com/notifications/unsubscribe-auth/AAQrr0iYDFFqCociL3jw-N_8je26F0Czks5sFsJigaJpZM4Nt93E .

[rgrunber]

Ok, how about the changes proposed below. I would either need permissions on the jetty job on ci.eclipse, or I could play around and show an example from some other HIPP under my control.

We create a new hudson job where we build on a daily or weekly basis. It consists of cloning jetty/org.eclipse.jetty.releng.bundles (modified to reference https://oss.sonatype.org/content/repositories/jetty-snapshots/), setting the version to what's under development (eg. 9.4.7) and building.

We could have some location like http://download.eclipse.org/jetty/updates/jetty-bundles-9.x/latest-9.4.x/ that gets cleaned out and the newer generated repository placed within.

The only change that would be needed on the jetty/org.eclipse.jetty.releng.bundles side would be to reference https://oss.sonatype.org/content/repositories/jetty-snapshots/ in the list of repositories. There would be no additional burden on the person doing the jetty releases as the 'latest' repo generation should be completely automated.

[joakime]

At this current time the following branches are all active and "under development"

• jetty-9.2.x (java 7 support version)
• jetty-9.3.x (http/2 initial support version)
• jetty-9.4.x (the HEAD/stable version)
• master (aka Jetty 10.x. no releases yet)

That means 4 HIPP jobs then, right?

[rgrunber]

Well on the Eclipse side, development would follow jetty-9.4.x as this is likely what the platform would continue to use (4.7.0 seems to be using 9.4.5). Even if the next major release committed to using Jetty 10, and the previous stuck to 9.4.x, that would be 2 HIPP jobs, assuming the older release would choose to update jetty at all.

[joakime]

Got it. The HIPP jobs should be focused on other Eclipse project consumers of the p2 repositories at the Eclipse side. 3rd party usage of the p2 repositories are not supported then?

[rgrunber]

It's possible but offhand I'm not sure if there's anyone other than Eclipse projects, that would consume from http://download.eclipse.org/jetty/updates/jetty-bundles-9.x/ .

Anyone could reference the p2 repositories for released versions, but the snapshot p2 repositories being proposed, at least initially, seem like something mainly to solve a problem for the Eclipse platform, hence the limited scope. However, if it all works, it's probably not that much work to create another job that just builds from another branch.

UPDATE: If you mean in the general sense, could the snapshot p2 repositories match the branches/released artifacts on maven central, then I don't think that even happens for released content. While 9.4.x and 9.3.x are up to date, jetty has a 9.2.22 release, but in http://download.eclipse.org/jetty/updates/jetty-bundles-9.x/ the latest, is 9.2.13.

[akurtakov]

I fully support this. Having latest (9.4.x now) p2 repo until the process is solid and can be extended to other versions after that.

[jmcc0nn3ll]

I am out most of this week but trying to keep up on this, am I understanding that @rgrunber is keen on taking point on this and open to shepherding the process? I think steps would be to get a bug on bugzilla opened up to get a new job created and a new eclipse based git repo setup for him to wire up whatever tycho process is required for what they need? Sounds like there is interest in getting it setup, just no one on the Jetty side has more then a passing familiarity to this stuff, happy to assist but I don't see any of us able to take lead and advocate for it.

[rgrunber]

Just to confirm, yes, I'm fine with starting work on this once the hudson job is created. I don't think there is a need for a new eclipse based git repo. The entire process can be done in the job configuration.

[sbordet]

@WalkerWatch can you please review this issue and close it if it's resolved ?

[rgrunber]

Has the CI job (at least 1 initially for testing) for doing this been created ? I think it was agreed that I would take care of implementing this but I've been waiting on a job to be created and have permissions granted to configure it.

[jmcc0nn3ll]

I don't think anything has been created for this within Jetty Eclipse CI, I half thought it would be done under whatever project was looking to manage or set this up.

@janbartel and @gregw does it make sense for us to just ceed control of the git repo that we originally setup for this whole p2 repository process over to these guys?

http://git.eclipse.org/c/gerrit/jetty/org.eclipse.jetty.releng.bundles.git/

That would let these folks do whatever they want to be able to generate the p2 repository for testing and overall publishing and consumption of these things outside of maven central. Seems like the responsibility on the Jetty side of things ought to strictly be getting released artifacts into Maven Central which is where our domain of expertise is.

thoughts? I mean it made sense for us to do all this stuff when we were a part of the release train but we have been so long off of that it isn't clear to me why we are point on P2 repositories at all anymore. Makes sense we do everything we can to produce good osgi bundles, but the p2 repositories themselves?

[janbartel]

@jmcc0nn3ll I'm happy to hand over the maintenance of the p2 release processes and the generation of the p2 releases - I'm sure whatever we're doing is antiquated as it was set up a long time ago and we just do enough to keep it ticking over. My faint concern is that we currently are picky about what we publish in the p2 world - we don't include bundles for everything that constitutes a jetty distribution, just a "reasonable" selection. The process of deciding what is a "reasonable" selection might be difficult for someone external to the jetty project?

[rgrunber]

I would guess that the only bundles being published in the p2 software sites are currently contained within : http://git.eclipse.org/c/gerrit/jetty/org.eclipse.jetty.releng.bundles.git/tree/jetty.bundles.f/feature.xml ? This would not change with the new process. We would continue to publish only what is specified in that feature.

[akurtakov]

I'll close this one for more dedicated issues if/when they arise.