search

eclipse-birt / birt

Eclipse BIRT™ The open source reporting and data visualization project.
http://www.eclipse.org/birt
Eclipse Public License 2.0
455 stars 389 forks source link

Improve the release engineering #1286

Closed merks closed 11 months ago

merks commented 1 year ago

Tasks

Description

The setup should produce a usable workspace amenable to contribution. The target platform should be modernized to use newer dependencies; ideally it should be easier to maintain it and to update it to newer dependencies in the future. It should be easier (possible) to do local Tycho builds. It should be easier to produce nightly, milestone, and release builds. There appear to be too many products (*.product) and too many update sites (category.xml). It's not clear why those are all needed and who all consumes those. Simplification and reduction of these would help streamline the releng process.

Perhaps the EPP Eclipse IDE for Java and Report Developers could be revived, but the build results are of a quality not consumable by SimRel:

https://download.eclipse.org/oomph/archive/reports-extra/birt-snapshot/download.eclipse.org/birt/update-site/snapshot/index.html

merks commented 1 year ago

I've been trying to get the tests to run locally, with some success but also some failure, but now I see the build uses -Dmaven.test.skip=true which is most disconcerting!

wimjongman commented 1 year ago

Do you mean the main build? I can imagine that because that one is only to publish the result. All PR checks are running with tests. E.g.:

https://github.com/eclipse/birt/actions/runs/4773353790/jobs/8486375628?pr=1271

wimjongman commented 1 year ago

https://download.eclipse.org/oomph/archive/reports-extra/birt-snapshot/download.eclipse.org/birt/update-site/snapshot/index.html

😳 I did not know we were in this bad shape.

merks commented 1 year ago

My branch that I was working on, unknown to me, was badly out of date. 😢 So I'm reworking the changes to get back to the point I was at yesterday. And there are many interrupts today we shall see how far I get today...

wimjongman commented 1 year ago

Thanks, Ed. No rush.

@eclipse/technology-birt-committers Please read this thread. We are going to improve our releng and with that our overall ability to execute.

merks commented 1 year ago

@wimjongman

To really use the latest of everything, e.g., Platform, GEF and so on, you'd need to use Java 17 for some things, e.g., for the JustJ JRE used for the products. Does anything speak against switching to Java 17 which is the latest LTS version?

wimjongman commented 1 year ago

We can go to Java 17.

wimjongman commented 1 year ago

We have been moved to our own organisation:

https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/3185

merks commented 1 year ago

FYI, I will wait for this IP review to complete

https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/8506

These are all the BIRT defined products:

image

These are all the BIRT defined sites:

image

How will "we" go about deciding which products and sites are import to keep and to publish?

wimjongman commented 1 year ago

@hvbtup @speckyspooky @claesrosell please comment on the above https://github.com/eclipse-birt/birt/issues/1286#issuecomment-1557276627 Which of the products and sites are used by your consumers?

speckyspooky commented 1 year ago

@wimjongman From our side the main products are:

claesrosell commented 1 year ago

We are not using any product, we are:

  • Consuming various Birt plugins via our target platform
  • Installing the "Birt Report Designer" -feature into Eclipse IDE.

so I guess we only use "org.eclipse.birt.p2.updatesite" from the list above.

csut4735 commented 1 year ago

@wimjongman From our side the main products are:

  • "BIRT Report Designer All-In-One"
  • "BIRT Engine Runtime" (OSGi)

That's all we use too.

hvbtup commented 1 year ago

We are using:

"BIRT Report Designer All-In-One"
"BIRT Engine Runtime" without OSGi
chloetz commented 1 year ago

We are using:

"BIRT Report Designer All-In-One"

"BIRT Engine Runtime" with OSGi

Von: Henning von Bargen @.*** Gesendet: Dienstag, 23. Mai 2023 13:39 An: eclipse-birt/birt Cc: Loetz, Christophe; Team mention Betreff: Re: [eclipse-birt/birt] Improve the release engineering (Issue #1286)

We are using:

"BIRT Report Designer All-In-One"

"BIRT Engine Runtime" without OSGi

— Reply to this email directly, view it on GitHubhttps://github.com/eclipse-birt/birt/issues/1286#issuecomment-1559120847, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ATFLNSBTTKPPQEADK7FPLCTXHSOWXANCNFSM6AAAAAAYJLVV5E. You are receiving this because you are on a team that was mentioned.Message ID: @.***>

merks commented 1 year ago

The following is an observation. This is what's published for the most recent release:

image

There is a single update site:

https://download.eclipse.org/birt/update-site/4.13.0/

So I think there are just two products there along with some other zips that I have yet to understand their content and purpose, and there is only a single update site...

So while the build produces many more products and site, these aren't published currently.

A goal here is to streamline the production of the site as well as any/all useful/necessary associated packaged results. I will make the assumption that things not currently leading to published results are not needed in the build and that no one is relying on building such things themselves directly. I want to state that assumption explicitly so that someone can contradict it if it's wrong.

merks commented 1 year ago

@wimjongman

BTW, it will be much easier for me/us if I have commit rights such that I can help with things such as target platform updates, enabling more tests, including managing their shared launch configurations so you can easily run them in the IDE, setting up build jobs and so on. I got most of the tests working now, except the UI tests, because I'm not sure they will run on the server without some configuration. Also, I gave up on this test because there are so many errors and failures and I don't understand well what they are testing:

image

speckyspooky commented 1 year ago

There are many many test based on the full history of BIRT. I don't know if you are able to understand all in full. This is a very hugh challenge.

The marked line is based on a special spudsoft-emitter UserProperty. With that property you can control the handling of empty rows of the spudsoft-excel emitter. And this test (my interpretation) check the correct working based on a fix in the past.

But I see many many more issue-testcases which would be good like standard test cases.

merks commented 1 year ago

Mostly I'm trying to make sense of the tests so I know how to launch each of them and which ones will need further attention from you experts. So far I have this set of launchers that I'm working on and will include in the Git Repo so that they are reusable:

image

But this is a work-in-progress...

merks commented 1 year ago

@wimjongman

I am under the impression that BIRT is working towards a 4.14 release. But the POM, feature, and bundle versions are all (mostly all) still 4.13. I want to manage updating these easily in a more automated way so I'm analyzing the current state of all the versions.

Once thing that seems a little concerning to me is the class/jar files in the Git repository:

I only noticed these because they don't have 4.13 versions, which makes sense. But one has to wonder where these come from and how they might be updated in the future, if/when necessary.

wimjongman commented 1 year ago

Hi Ed @merks. Thanks for fixing this. I always did this manually. I have a PR pending with this fix but I could not get the build going using the latest Eclipse deps.

wimjongman commented 1 year ago

Regarding the comment https://github.com/eclipse-birt/birt/issues/1286#issuecomment-1575497550. Maybe you can create a new issue out of this.

merks commented 1 year ago

I thought I would go insane if I had to do it manually. So tedious and error prone. I have a simple Java class that does all the work so next time will be very easy...

merks commented 1 year ago

I have the publishing/promotion step working locally:

image

The JustJ publisher is enhanced so one can specify a list of "download artifacts" and these will be maintained along with the p2 repository itself. They're physically needed in a "downloads" folder within the repository folder like this:

image

I carefully analyzed was being "published" currently to determine where it came from and ensure that these same things are published with the new approach.

Any thoughts, questions, or concerns?

As soon as I have all my permissions in place I'll create a Jenkinsfile for a job that drives this...

ruspl-afed commented 1 year ago

I have a simple Java class that does all the work so next time will be very easy...

Do you a have another class to help with PR review? 😊

ruspl-afed commented 1 year ago

Any thoughts, questions, or concerns?

I have positive impression from using this magic. Two things to mention: 1) for "release" builds it is not yet clear for me how it could be "tailored" with legacy releases that project already has. 2) cleanup procedure for old nightly and stable builds is not yet clear for me, since standard "archive" UI is overlapped by generated index.html pages

merks commented 1 year ago

I did a fake old release on one of the projects just by coping a release repository folder to the correct/expected location with the correct/expected name. I could try that again, but in this case, it's much trickier with all these special download artifacts because those are recorded as repository properties during the promotion process, so it's not easy to manage applying it to an older release. Do you think that's important or can we make a fresh start in the new location leaving the other stuff where it is right now for a while?

Nightly builds are self cleaning. The -retain 5 argument specifies the cutoff number to retain. The milestone builds are also self cleaning. As soon a milestone is published with a higher branding version -version-iu org.eclipse.birt.feature all the current milestones are deleted and that new one with the new higher version becomes the only child at that point, so assuming that one doesn't build milestones for years without increasing versions, that should be limiting. If necessary an option to limit the number could be implemented.

The retention policy is documented on the sites:

This composite update site references the 4 most recent successful nightly builds

This composite update site references x successful milestone builds. These sites will be retained only until the first successful milestone build of the next Xyz release.

speckyspooky commented 1 year ago

Question to the new build process. I have seen that the new version avoid the creation of the test segments like before. So all tests will be handled during the complete build process. These test segments had 2 functions:

  1. organize the test process into chapters and so we get an structured test organization
  2. the next segment will only be started if the segements are done before successfully

On my failed build of the weekend I saw that the build process go through till the end, create all zip-files and so on. But at the end the process figured out that there was an test failure.

At the end the full build process runs longer like the version before due to the changed error handling and we loss the test segmentation.

Therefore my questions:

The important topic for me is the failure handling that the process tries to stop after an test failure was detected.

speckyspooky commented 1 year ago

I forgott to say, I like the improvement of the build process and also the enhancements of the test cases!

merks commented 1 year ago

I did add --fail-at-end here so that all the test run:

https://github.com/eclipse-birt/birt/blob/3b72e4b9f121433508e5fa1eec1a5d2603844b01/.github/workflows/ci.yml#L46

This can also cause things to run that otherwise wouldn't run at all. That can of course be removed and then the build will stop as soon as one test fails. That has the advantage that the first failure can be noticed more quickly and the disadvantage that if 5 different test suites/projects have failures, you won't notice a second failure until you've fixed the first one so you might need to do 5 builds before you get that far (which is why I changed it so that I could run the full build and know about all the failures while testing the huge number of changes that I had made.) But it's probably good to change it back again now. Note that without the "segmentation" in the ci.yml, the tests run much earlier in the build; in the past they ran after the full build...

wimjongman commented 1 year ago

Do you think that's important or can we make a fresh start in the new location leaving the other stuff where it is right now for a while?

We should make a fresh start.

We can archive all the old bits. The redirection that the foundation has made will make sure that download requests are forwarded to the archives.

I have archived everything except the last two releases already and had little complaints. So from 4.14 onwards, we can use the new approach/website.

merks commented 1 year ago

FYI,

The first published builds are available at this temporary location:

https://download.eclipse.org/birt/updates-tmp/nightly/latest/

I want to ensure that the whole end-to-end processing, including creating a release, produces the correct structure. Eventually I will delete the entire updates-tmp folder and begin producing build results in the updates folder. All the links appear to be working correctly...

wimjongman commented 1 year ago

That looks so great!!!

I added a subtasks to the top comment so we don't forget some housekeeping.

wimjongman commented 1 year ago

I did a smoke test on the latest build, and it works without issues.

speckyspooky commented 1 year ago

Sounds very good!

merks commented 1 year ago

I've tested the whole process, so there are milestones and releases here now:

https://download.eclipse.org/birt/updates-tmp/

Also, I'm migrated the 4.12.0 and 4.13.0 releases into the new structure so those are present as well.

If no one has any concerns or significant changes they'd like to see, I will now build into the updates folder and will migrated the 4.12.0 and 4.13.0 releases into the folder as well. OK?

Of course I won't produce a release build for 4.14.0 until we are ready for that!

wimjongman commented 1 year ago

Yes, great! Please let me know what the new address will be.

I think we are ready for 4.14

@eclipse-birt/technology-birt-committers @eclipse-birt/technology-birt-project-leads any concerns?

merks commented 1 year ago

Before we produce a release, I'd like to figure out why there are so few proper licenses being used and also where the unsigned content comes from:

https://download.eclipse.org/oomph/archive/reports-extra/birt/download.eclipse.org/birt/updates-tmp/release/4.14.0/index.html

I know some unsigned content problems come from very old Orbit bundles jar-signed with weak algorithms that are no longer recognized as jar signed, but that's not the case for all of them.

merks commented 1 year ago

FYI, the new update sites have been built at the proper location:

https://download.eclipse.org/birt/updates/index.html

The sites include those for the two previous releases:

These p2 update sites will now always be available:

The sites are self-documenting. Note that each site provides a link to the git commit for that site's build.

These builds are produced by this multi-branch pipeline:

https://ci.eclipse.org/birt/job/build/

I expect this build will also kick in for PR verification, but of course in that case, it will not publish p2 repositories and will not sign content.

speckyspooky commented 1 year ago

The new download area looks very great. Is it possible to link the archive folder too?

So we would have all together.

merks commented 1 year ago

Anything is possible...

That being said, things are in the archive for a reason, i.e., because we don't generally expect nor want people to be using such very old things when we have new-improved-and-better things.

There's a lot of old stuff here:

https://archive.eclipse.org/justj/?file=birt/downloads

What kind of links specifically did you have in mind? I.e., where would such links appear, to what would they link, and why would a user need (to know) that? (And to what does "all" refer to in "have all together"?)

csut4735 commented 1 year ago

Rather than moving old stuff to (yet another) new place, I would prefer you leave it where it can be found by people using old links, and insert a link to the latest download directory there.

-- Colin

On 8 Jun 2023, at 4:52 pm, Ed Merks @.***> wrote:



Anything is possible...

That being said, things are in the archive for a reason, i.e., because we don't generally expect nor want people to be using such very old things when we have new-improved-and-better things.

There's a lot of old stuff here:

https://archive.eclipse.org/justj/?file=birt/downloads

What kind of links specifically did you have in mind? I.e., where would such links appear, to what would they link, and why would a user need (to know) that? (And to what does "all" refer to in "have all together"?)

— Reply to this email directly, view it on GitHubhttps://github.com/eclipse-birt/birt/issues/1286#issuecomment-1581987962, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A2TG64JTVIM6SYXNAC7R3FDXKFZDHANCNFSM6AAAAAAYJLVV5E. You are receiving this because you commented.Message ID: @.***>

speckyspooky commented 1 year ago

I haven't said that the old link will be deactivated. I prefer 1 place to find all. It makes no sense to have 2 incomplete places/download areas.

It would be nice to have also on the new area the archive links.

speckyspooky commented 1 year ago

@merks I would say to have by default a link to the current archive. My favorit would be an archive area with the links of all old versions (releases) and there according sources.

merks commented 1 year ago

@csut4735

I did not move anything from anywhere to anywhere.

I think maybe you are suggesting that perhaps

https://download.eclipse.org/birt/update-site/snapshot/

could be a composite that references:

https://download.eclipse.org/birt/updates/nightly/latest

and that

https://download.eclipse.org/birt/update-site/latest/

could be a composite that references

https://download.eclipse.org/birt/updates/release/latest

So that those will just continue to work But I'm reading between the lines. In any case, the new structure cannot "just be moved" into the old poorly-manged structure...

@speckyspooky

Similarly, I asked 3 specific questions and I see no answers. The "new area" is a whole web of pages so please be explicit about "this page should have this link to this thing because...". I.e., show the link and state where (on which page) it is located. Also, you talk about something being incomplete, but I don't know what is incomplete. I assert that the new locations are 100% complete and contain everything in the old locations. And I really mean everything, so if there is a thing that you believe is missing/incomplete, please be explicit what that thing is.

speckyspooky commented 1 year ago

I will give later a detailed summary. The point is that some users use older versions from archive. That was for example the reason why the archive link with the old versions was created in the current download page.

My supplier of a specific Bird-framing use version 4.10-osgi, so I need a link to the old version.

wimjongman commented 1 year ago

Hi. I agree with @merks to NOT point to the archive in the stuff that Ed is building for us. The Archives will still be there prominently on the website but it should not be part of our current downloads pages.

So

  1. The Archives will stay where they are: https://archive.eclipse.org/birt/downloads/drops/
  2. The archives are featured on our main page: https://eclipse-birt.github.io/birt-website/
  3. The stuff that Ed is building does not carry any old stuff.

Good compromise?

merks commented 1 year ago

Indeed, I see now that we were (probably) talking about the two links on this page:

https://eclipse-birt.github.io/birt-website/

That's the second checkbox at the top of this issue that Wim added the other day. And yes, that will certainly need proper consideration before we are complete here! I imagine that, as Wim suggest, archive will continue to point where it does now. And that downloads will point here:

https://download.eclipse.org/birt/updates/

where the user can navigate to whatever they are most interested and can get at the downloads if they wish, e.g., here the latest release, which is currently 4.13.0 and provides the linked downloads:

image

wimjongman commented 1 year ago

Unless you feel strongly about it, I suggest replacing 'glyph' with 'icon'. Technically you are correct but do normal people know what glyphs are?

speckyspooky commented 1 year ago

And know my question to the last screen, why we shouldn't have on the lefthand side also a chapter "archive"? We have Release, Milestone, etc. but we avoid "archive" here.

From point of user view it would make sense, from point of developing it is not necessary.

But my thinking is we don't improve it only for us.