Retrospective for April 2021 Releases

[adamfarley]

Date: Monday, 26th of April, 2021

Time: 4pm BST, 11am Ontario.

This will be a Slack call after the release, with at least a week of notice in the #Release Slack channel.

A bulleted list of all proposed agenda items in this issue (whether at the top or in a comment) will be created on the day of the call, with a list of actions added at the end.

Invited: @AdoptOpenJDK/tsc

Retrospective Owner Tasks (in order):

• [x] Post this issue's URL on the #Release Slack channel around the start of the new release.
• [x] Copy actions from the previous retrospective into this issue, while ignoring actions that have an issue link (i.e. clearly-completed actions).
• [x] Wait until all builds have been released, with no clear signs of additional respins.
• [x] Announce the retrospective Slack call's date + time on #Release at least one full week in advance, and send out meeting invites.
• [x] On the day of the call, compile all of the agenda items into a tick-box list inside a single comment.
• [x] Host the slack call for the retrospective, including:
  • [x] Iterating over the actions from the previous retrospective issue, annotating both issues with any issues that have been raised for them.
  • [x] Iterate over the agenda tick-list, ensuring everything gets debated.
  • [x] Create a clear list of actions at the end of the retrospective, including volunteer names.
• [x] Close the issue for the previous retrospective, ensuring that every action is annotated with its conclusion (issue link, ignored, etc).
• [x] Create a new retrospective issue for the next release.
• [x] Set yourself a calendar reminder so that you remember to commence step 1 (in the new issue) just before the next release.

Add proposed agenda items as comments below.

[smlambert]

From https://github.com/AdoptOpenJDK/TSC/issues/227#issuecomment-824307394, relating to extended.openjdk tests on Windows:

Notes for release retrospective (from Slack conversations):

• machines have old processes laying around that are not being addressed by a process clean up that was added here
• we had previously tried to raise an upstream issue to address issues of processes not being cleaned up by tests, response was that it should be up to the runners to take care of that
• when leftover processes are running, new jobs get blocked and run to the full timeout factor, leaving us to hit the default timeout of 10hrs then abort without completing
• https://github.com/adoptium/ci-jenkins-pipelines/issues/109 was raised a while ago which will alleviate some of the timeout issues (that are side-effects of the core issue of processes left laying around)

Smoke tests are 'new', many issues have been cleaned up this week, and show that they will be useful at the very least in clarifying which features should be present for which platforms (good example with JFR on AIX).

[karianna]

• Need to disable all nightly builds including docker multi-arch and any other misc builds.
• Windows Hosts Running out of Disk Space - Plan ongoing
• PPC AIX Missing Deps - Fixed
• Xvb failure on Linux ARM32 - May need to upgrade or remove plugin
• Need to fix Lots of 16 test failures

[karianna]

The Git Workspace Corruption issues are a combination of old processes lying around this should resolve it and the fact we have old jenkins agents/git plugins (will be resolved in the new Jenkins rebuild)

[karianna]

Add a reenable nightly and weekly builds task to the chekclist

[adamfarley]

Actions from last retro:

• [x] No actions

Agenda for this retro:

• [x] (Shelley) machines have old processes laying around that are not being addressed by a process clean up that was added here
• [x] (Shelley) we had previously tried to raise an upstream issue to address issues of processes not being cleaned up by tests, response was that it should be up to the runners to take care of that
• [x] (Shelley) when leftover processes are running, new jobs get blocked and run to the full timeout factor, leaving us to hit the default timeout of 10hrs then abort without completing
• [x] (Shelley) adoptium/ci-jenkins-pipelines#109 was raised a while ago which will alleviate some of the timeout issues (that are side-effects of the core issue of processes left laying around)
• [x] (Shelley) Smoke tests are 'new', many issues have been cleaned up this week, and show that they will be useful at the very least in clarifying which features should be present for which platforms (good example with JFR on AIX).
• [x] (Karianna) Need to disable all nightly builds including docker multi-arch and any other misc builds.
• [x] (Karianna) Windows Hosts Running out of Disk Space - Plan ongoing
• [x] (Karianna) PPC AIX Missing Deps - Fixed
• [x] (Karianna) Xvfb failure on Linux ARM32 - May need to upgrade or remove plugin
• [x] (Karianna) Need to fix Lots of 16 test failures
• [x] (Karianna) The Git Workspace Corruption issues are a combination of old processes lying around this should resolve it and the fact we have old jenkins agents/git plugins (will be resolved in the new Jenkins rebuild)
• [x] (Karianna) Add a reenable nightly and weekly builds task to the checklist

[adamfarley]

Actions for this retro:

• To add to the pre-release section as discussed. (Volunteered by team: Andrew?)
• To make sure Haroon's fix for https://github.com/AdoptOpenJDK/openjdk-infrastructure/issues/2149 is reflected in an ansible playbook. (Volunteered: Haroon, Sxa)
• To see if a Jenkins job should be created to establish confidence that a machine is ready to deploy (Shelley's "Weird Test" Job) (Volunteer: Shelley)
• To make Simon a committer to the openjdk-tests repo (Volunteer: Shelley)
• To throw "Solaris builds concern" onto the next PMC agenda. (Volunteer: Shelley?)

[andrew-m-leonard]

Yes, i'll take a look at this: To add to the pre-release section as discussed.

[adamfarley]

Closing. Actions ported to https://github.com/adoptium/adoptium/issues/77.