search

eclipse-windowbuilder / windowbuilder

Eclipse Windowbuilder
https://projects.eclipse.org/projects/tools.windowbuilder
Eclipse Public License 1.0
78 stars 30 forks source link

1.12.0 Releng #438

Closed wimjongman closed 1 year ago

wimjongman commented 1 year ago

A document to catch releng issues for 1.12.0

ptziegler commented 1 year ago

I'm not overly familiar with the release train.

My only concern would be our GEF classic dependency on the upcoming 3.16, which is announced for release together with the Eclipse 4.28 on the 14.06.2023. I think it makes sense to then do have a joined release.

merks commented 1 year ago

You're scaring me because you are on the release train:

image

GEF makes regular contributions:

image

It seems a significant problem that WindowBuilder doesn't produce stable milestone builds. Hence you talk about creating a release, but the normal process is to produce stable milestone builds that are contributed to SimRel regularly on schedule. Eventually release candidate builds are produced (late this month) and when all looks good, you produce a release synchronized with the SimRel.

This is why I helped Mylyn set up such a repo structure which is automatically maintained by the builds:

https://download.eclipse.org/mylyn/updates/index.html

I am doing the same for Data Tools. Maybe you need to that too. A multi-branch pipeline would good too...

wimjongman commented 1 year ago

Ah, we are 😁😂 I forgot!

Ok, we are prepping the 1.12 release.

Currently, we are deploying our milestone into the lastgoodbuild repo.

Yes, it would be great if we had the same for WB.

This is our current dl site:

https://www.eclipse.org/windowbuilder/download.php

merks commented 1 year ago

The WB site will look "exactly" like these:

https://download.eclipse.org/oomph/dtp/updates/ https://download.eclipse.org/mylyn/updates/

This would be the "root" folder where the builds would published:

https://download.eclipse.org/windowbuilder/updates

You can of course keep your download page, but for the new stuff it would point into these location locations...

Are you @wimjongman @ptziegler okay with all that?

Are you in a hurry to publish a 1.12 release? Do you plan another release before the June SimRel release?

I think it would be better to use and "test" a new infrastructure for this purpose and I would prefer to see a milestone build published and contributed to SimRel first, that's why I ask if you're in a hurry...

I don't think there is more than a day or two of work involved to get this in place...

wimjongman commented 1 year ago

Yes, perfect. No, we are not in a hurry. This is the simrel release we want to publish.

We now have a current and a lastgoodbuild repo. Can we keep those? Otherwise, it is again extra work for the marketplace.

merks commented 1 year ago

The structure is analogous to this once the first release is published:

These are automatically kept up to date as builds are published. If you have some existing location that you'd like to keep, you can always make that location a composite that points at one of these new locations. The nightly repos are limited in number and old repos cleaned up automatically. The milestone build repos are cleaned up once a new version starts; the version is derived from the/a feature's version. The release repos are version numbered and are retained permanently.

I'll write some more centralized documentation once we get this in place...

merks commented 1 year ago

@wimjongman @ptziegler @lcaron

This is the branding image currently used for WB features:

wb-icon

Sorry, to say but it looks terrible.

How about we create a new one based on this SVG?

wb

That renders as a 32x32 PNG like this:

wb

I think this will look nicer among these:

image

I propose to put them here:

image

What do you folks think?

FYI, I'm making good progress on the build promotion infrastructure and have it working locally already:

image

merks commented 1 year ago

BTW, I noticed that I don't have a clone of this:

https://github.com/eclipse/windowbuilder-website

So I'll need to update the setup because it's cloning the Gerrit repository...

Have you considered asking for and migrating too an eclipse-windowbuilder organization?

wimjongman commented 1 year ago

Thanks for the branding logo idea. How about we ditch the gear and get a bit recursive?

wbbigwin

wblogo

wimjongman commented 1 year ago

Have you considered asking for and migrating too an eclipse-windowbuilder organization?

I will set this in motion. We need backup of our beloved project lead @lcaron for this as well.

wimjongman commented 1 year ago

Have you considered asking for and migrating too an eclipse-windowbuilder organization?

I will set this in motion. We need backup of our beloved project lead @lcaron for this as well.

I filed https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/3149

merks commented 1 year ago

The first nightly build is published from a test build so things are looking promising once we have permission to merge PRs again:

https://download.eclipse.org/windowbuilder/updates/

I think the bundles should be consistently prefixed with WindowBuilder in their name:

image

OK?

I tested installing all the bundles into an Eclipse SDK and the only concern is that several Nebula bundles were not available. We might consider providing a "dependencies" feature for pulling things into the site within strictly including a specific version in any specific feature, or adding an update site link to the category.xml, e.g, perhaps a link to this

https://download.eclipse.org/nebula/releases/latest/

WDYT?

wimjongman commented 1 year ago

You say within but you mean without a strict version?

I think adding the link to the latest of nebula is best.

ptziegler commented 1 year ago

Oh wow, that's really looking great so far!

merks commented 1 year ago

One thing that bothers me now is the issue I mentioned in https://github.com/eclipse-windowbuilder/windowbuilder/issues/438#issuecomment-1547962517 where I think linking to another repo is likely not a great solution because those links are permanently in place and that assumes that the folks at the other end up the link will always put reasonable content there. One can never assume anyone else does only reasonable things. 😞

Furthermore, I see there are already includes of 3rd party bundles, e.g.,

image

So the missing things have simply been overlooked...

What I would like to prototype is creating a new separate feature that includes all the 3rd party libraries. That feature will be included in the category.xml so that the 3rd party libraries are included in the site, but no actual user-installed feature includes specific versions and the bundles allow version ranges such that newer versions of these 3rd party could be installed yet some actual version is always available so that WB can be installed all by itself...

Hopefully everyone is okay with that approach...

merks commented 1 year ago

The experiment is working well. I've tested (with Oomph) that all of WB's feature install in an Eclipse SDK:

image

The only downside is that the user can see the feature in a category:

image

But I could probably enhance the JustJ publisher to remove this category to eliminate that problem...

wimjongman commented 1 year ago

Sure, that is fine as well! Thanks, Ed.

merks commented 1 year ago

I noticed there are no source features/bundles in the repository. Perhaps no one ever extends WindowBuilder so no one really needs such artifacts. Is that a correct assumption?

wimjongman commented 1 year ago

We never had a request, but it would be nice if they were available.

merks commented 1 year ago

The new multi-branch pipeline build is functional:

https://ci.eclipse.org/windowbuilder/job/build/

It kicks in for PRs and does a nightly build, promoting it to the update site, for every commit to master.

wimjongman commented 1 year ago

Wow. That would have taken us a long time to fix. Thanks for the help, Ed. It is a really good quality improvement.

merks commented 1 year ago

@wimjongman

If I understand correctly, https://download.eclipse.org/windowbuilder/latest/ will be a composite that points at https://download.eclipse.org/windowbuilder/updates/release/latest once that exists. Similarly, https://download.eclipse.org/windowbuilder/lastgoodbuild/ should already now be a composite that points at https://download.eclipse.org/windowbuilder/updates/nightly/latest which already exists. Then the https://ci.eclipse.org/windowbuilder/job/org.eclipse.windowbuilder/ job can be deleted and also the https://ci.eclipse.org/windowbuilder/job/Release/ job can be deleted. That leaves only the https://ci.eclipse.org/windowbuilder/job/build/ job.

Is that all correct?

wimjongman commented 1 year ago

The task of the release job was to promote a build to a release. How does this work with the new setup?

merks commented 1 year ago

The drop down has three choices, one of them is "release":

image

As the message suggests, it promotes the latest milestone build which has been well tested and has been contributed to SimRel to ensure that it aggregates with all the other SimRel content.

wimjongman commented 1 year ago

Ah, I missed that. Thanks for the clearing this.

@wimjongman

If I understand correctly, https://download.eclipse.org/windowbuilder/latest/ will be a composite that points at https://download.eclipse.org/windowbuilder/updates/release/latest once that exists. Similarly, https://download.eclipse.org/windowbuilder/lastgoodbuild/ should already now be a composite that points at https://download.eclipse.org/windowbuilder/updates/nightly/latest which already exists. Then the https://ci.eclipse.org/windowbuilder/job/org.eclipse.windowbuilder/ job can be deleted and also the https://ci.eclipse.org/windowbuilder/job/Release/ job can be deleted. That leaves only the https://ci.eclipse.org/windowbuilder/job/build/ job.

Is that all correct?

Yes.

merks commented 1 year ago

@wimjongman @ptziegler

I'm pretty much done here...

With a bit of trickery, I mirrored the 1.11.0 release into the new structure

image

So https://download.eclipse.org/windowbuilder/latest/ already composes https://download.eclipse.org/windowbuilder/updates/release/latest and https://download.eclipse.org/windowbuilder/lastgoodbuild/ composes https://download.eclipse.org/windowbuilder/updates/nightly/latest as discussed.

The latest milestone https://download.eclipse.org/windowbuilder/updates/milestone/latest is a 1.12 build:

image

I've updated the download page accordingly:

https://www.eclipse.org/windowbuilder/download.php

It would be nice if in the future we don't actually need to update the download page each release given the sites are self-documenting...

One of your marketplace listings https://www.eclipse.org/setups/marketplace/?id=4961416 was broken:

image

If everyone is okay, I propose to delete these two repositories:

image

And also these two jobs:

image

This job is quite handy and provides a slow poor-man's shell access to the download server:

https://ci.eclipse.org/windowbuilder/job/genie.windowbuilder-ssh/

Is there anything I've overlooked?

wimjongman commented 1 year ago

+1 for the deletes.

merks commented 1 year ago

@wimjongman @ptziegler

Are things in a good state to build a milestone and contribute it SimRel for m3? We have until Wedneday at the latest to do so. I've already contributed this one last week to see if aggregation works well with that:

https://download.eclipse.org/windowbuilder/updates/milestone/S202305170649

ptziegler commented 1 year ago

As far as I can tell, the master is stable.

Though ideally, I'd like to get rid of CGLib before the new milestone. Assuming that #458 is successful, there are only one or two classes in the Swing bundles left, which I should hopefully get done today or tomorrow.

merks commented 1 year ago

There’s no hurry. Just drop a note here when you’re done.

ptziegler commented 1 year ago

Well... CGLib is no longer used by WindowBuilder, but we sadly can't get rid of it, because it's a dependency of Digester3. Which I'm not yet sure how to deal with. But that shouldn't be an issue for the upcoming milestone.

merks commented 1 year ago

And that thing is old as the hills, not having produced something new since 2011:

https://commons.apache.org/proper/commons-digester/

😱

ptziegler commented 1 year ago

And that thing is old as the hills, not having produced something new since 2011:

It really is... Even though their GitHub page is still somewhat maintained, the last issue was fixed back in 2020. In any case, that's not a can of worms I want to open so soon after the ByteBuddy migration.

https://github.com/apache/commons-digester

Given that the dependency to CGLib was introduced with Digester 3.2, my current idea is to build WB against the 3.1. Without an upper bound in the Manifest files, users are free to install a newer version, if they want. But then they have to contribute the CGLib bundle.

Which then leads to the next problem, which is that EasyMock also has a dependency on CGLib. So I have to see whether we can update the embedded jar to a newer version or switch to Mockito there as well.

In short, not something that's done by the upcoming milestone... :sweat_smile:

merks commented 1 year ago

@ptziegler

Are we ready to do a milestone build now? We need one today.

ptziegler commented 1 year ago

@merks We're good to go.

merks commented 1 year ago

@ptziegler

The build failed with two test failures:

https://ci.eclipse.org/windowbuilder/job/build/job/master/19/

But it looks like it built the same revision has this:

https://ci.eclipse.org/windowbuilder/job/build/job/master/18/

I'll kick it again and keep my fingers crossed. 🤞

merks commented 1 year ago

The finger crossing seems to have helped so now we have this:

https://download.eclipse.org/windowbuilder/updates/milestone/S202305241100

I'll contribute that to SimRel now...

wimjongman commented 1 year ago

Cheers! 🎉

merks commented 1 year ago

@ptziegler

The builds are currently timing out so I am not able to produce a milestone build for contribution today.

https://ci.eclipse.org/windowbuilder/job/build/job/master/

If this gets fixed by tomorrow morning, I could still do that.

Keep in mind that the final contribution, RC2, is due next Wednesday, so please focus on stability and release-quality over the next week so that we have a release candidate ready to promote as the 1.12.0 release.

ptziegler commented 1 year ago

Hm... I just ran the tests locally again and they roughly took 20 minutes. Which is about what they always took over the past few weeks. I find it weird that there is such a huge discrepancy between my machine and the build node.

And the big time consumer is primarily the Java parser with ~8min, which I haven't really touched at all. If it helps with the milestone build, I have no issue rolling back e1d643b494996f085768cc7606d3961c01133265 and 4dde29e9d8c2b2a4fe3677985a2eab302708db1c.

I'll try to have a closer look at the profiler results over the next few days. Perhaps I can figure out what's going on here.

merks commented 1 year ago

@ptziegler

Yes, that seems strange indeed! I'll monitor how the nightly build completes and if that's okay, I'll kick off a milestone build and contribute it for RC1. Thanks for the speedy response! 🚀

merks commented 1 year ago

Unfortunately the builds just seem to hang producing endless entries like this in the log:

(SWT:1599): Gdk-WARNING **: 06:03:03.119: A paint operation on the window is alredy in progress. This is not allowed.

(SWT:1599): Gdk-WARNING **: 06:03:03.121: gdkwindow.c:3014: no preceding call to gdk_window_begin_draw_frame(), see documentation

I'm not able to wait longer for an RC1 contribution. 😢

ptziegler commented 1 year ago

Unfortunately the builds just seem to hang producing endless entries like this in the log:

(SWT:1599): Gdk-WARNING **: 06:03:03.119: A paint operation on the window is alredy in progress. This is not allowed.

(SWT:1599): Gdk-WARNING **: 06:03:03.121: gdkwindow.c:3014: no preceding call to gdk_window_begin_draw_frame(), see documentation

I'm not able to wait longer for an RC1 contribution. 😢

This warning is thrown whenever one of our tests is calling getImageSurface() in native code (i.e. the rcp.c). The fact that it is shown over and over again seems to indicate that the tests are still running, it's just not shown in the log.

As a dirty workaround, could you try to increase the timeout in the org.eclipse.wb.tests/pom.xml file? It's currently set to 1h. If it's just the tests taking a long time, perhaps we can increase it a little until we figure out what's wrong.

merks commented 1 year ago

This build is running with a 90 minute timeout:

https://ci.eclipse.org/windowbuilder/job/build/job/master/27/console

ptziegler commented 1 year ago

Hm... Though I wonder why the tests are timing out, even after reverting the recent revisions. I get the feeling something isn't running smoothly on the Jenkins.

merks commented 1 year ago

Yes, it's very odd especially because for your PR the Jenkins build also runs and it does not timeout. The PR build also does not do signing and that's definitely relatively slow. Let's see how this goes.

ptziegler commented 1 year ago

[Pipeline] timeout Timeout set to expire in 50 min

The Jenkins build itself also has a timeout of 50min. So it might really be a combination of slow tests + slow signing that's causing problems.

This would mean that even with an increased timeout, the build is likely to still fail because of it. Who here has the permissions to edit this job? I sadly don't.

merks commented 1 year ago

Note that after 34 minutes it is already in the "cycle" of logging the same message over and over...

merks commented 1 year ago

Now the build is finished in 37 minutes. What to conclude from that? 😱

ptziegler commented 1 year ago

Note that after 34 minutes it is already in the "cycle" of logging the same message over and over...

I think that's "normal" behavior, because we're using a deprecated Cairo API. As I mentioned earlier, the warning is shown whenever we create a snapshot. So effectively once for each and every SWT/RCP test. It would probably be clearer if JUnit would also log, which test it's currently executing...

Now the build is finished in 37 minutes. What to conclude from that? 😱

Hm... So there really is something slowing down the tests every now and then. In general, we should probably increase the timeout on the Jenkins job, given how tight the limit seems to be.

I also had a look at the profiler results this morning, and I think I missed a few optimizations which should shave off a couple of minutes when running the tests.