Modify inconsistent messages with list of features logged on server start

[brenthdaniel]

At server start, the feature manager will log a list of added features in a CWWKF0012I audit message and removed features in a CWWKF0013I audit message. In the case of a warm start, the feature manager will instead log the entire list of installed features in the CWWKF0012I message: [AUDIT ] CWWKF0012I: The server installed the following features: [el-3.0, jsp-2.3, servlet-3.1].

This has lead to confusion in the past from users who expect the CWWKF0012I to always output the full list of installed features. In the case where the list of features is updated in between server starts, someone could believe that previously installed features are no longer there.

It would be nice to have more consistency here, but any change here has a strong potential to break any automation that depends on the CWWKF0012I message.

Solution 1: Add a new feature manager message to log the list of all installed features and only issue CWWKF0012I for newly added features. This would provide consistency and still provide a list of all installed features to anyone depending on it. This change would have a high probability of breaking anyone that relies on the feature messages.

Solution 2: Change the CWWKF0012I message to always output the full list of features on server start. When the list of features is updated in a running server, continue to list only the delta. Maintain the CWWKF0013I message for any features that were removed at startup. This is much less likely to break automation. We lose some information here on the feature delta, but that may not be important.

Solution 2b: The same as solution 2, but don't issue CWWKF0013I message for features removed at startup.

[frowe]

Could you clarify the description "At server start, the feature manager will log a list of added features in a CWWKF0012I audit message and removed features in a CWWKF0013I audit message." Which kind of server start are we talking here? What would be the use case for monitoring CWWKF0012I that we envision possibly breaking automation? From the description and the slack discussion, it seems this issue only appears if you edit the server config while the server is stopped? That seems common in a dev env, but where else? If only dev, it doesn't seem likely we're breaking automation. In a container env, I would assume we're always seeing the full feature list since we're creating the server and not restarting it?

[brenthdaniel]

Solution 1 would immediately break our FAT tests that rely on looking for specific features in the 12I message. That's not an issue for us given that we can just fix it, but it's reasonable to think that embedders could have similar logic. It's less likely that end users would have similar log scraping, but would not be surprising.

Solution 2 would be much less likely to break automation as it would only affect cases where a test was specifically looking for a delta.

Here is every server start scenario: Start a new server with jsp-2.3 enabled (aka, cold start): [AUDIT ] CWWKF0012I: The server installed the following features: [el-3.0, jsp-2.3, servlet-3.1]. Stop the server. Edit server.xml to add jdbc-4.0: [AUDIT ] CWWKF0012I: The server installed the following features: [jdbc-4.0, jndi-1.0]. Stop the server. Start the server (aka warm start). [AUDIT ] CWWKF0012I: The server installed the following features: [el-3.0, jdbc-4.0, jndi-1.0, jsp-2.3, servlet-3.1]. Edit server.xml to add servlet-4.0:

[AUDIT   ] CWWKF0012I: The server installed the following features: [servlet-4.0].
[AUDIT   ] CWWKF0013I: The server removed the following features: [servlet-3.1].

Stop the server. Edit server.xml to remove servlet-4.0:

[AUDIT   ] CWWKF0012I: The server installed the following features: [servlet-3.1].
[AUDIT   ] CWWKF0013I: The server removed the following features: [servlet-4.0].

[brenthdaniel]

Various permutations of solutions were discussed. Additional solutions:

3. Keep 12 and 13, but add the full list of features to the end of the message. Something like:

   [AUDIT   ] CWWKF0012I: The server installed the following features: [servlet-4.0]. The current list of installed features is: [el-3.0, jdbc-4.0, jndi-1.0, jsp-2.3, servlet-4.0].
   [AUDIT   ] CWWKF0013I: The server removed the following features: [servlet-3.1]. The current list of installed features is: [el-3.0, jdbc-4.0, jndi-1.0, jsp-2.3, servlet-4.0].

4. Leave 12 and 13 as they are and add a new message that always lists all features. This is similar to (1) but wouldn't break any automation because 12 would stay the same. The down side is that, in most cases, there would be two lines that show all installed features, eg:

   [AUDIT ] CWWKF0012I: The server installed the following features: [el-3.0, jsp-2.3, servlet-3.1].
   [AUDIT ] CWWKF9999I: The current list of installed features is: [el-3.0, jsp-2.3, servlet-3.1].

Also, if we end up making changes to the messages, we might want to change 'installed' to something else like 'added' or 'enabled'.

[brenthdaniel]

The requirement from the embedder that raised this issue is to have the full list of active features logged during server start for service reasons, but it's not important whether it's in a new or existing message.

[brenthdaniel]

At the 3/29 meeting we decided to pursue the following:

1. The CWWKF0012I and CWWKF0013I messages will remain as is, except that CWWKF0012I will now always list all features.

2.A new mesage CWWKFxxxxI will be added that will only be output when there is a feature delta and will list only added features.

No changes will be made to change 'installed' to 'added' in the existing CWWKF0012I message.

We will check with the maven/gradle tools and WDT before making any changes.

[hlhoots]

Description

A new informational message CWWKFxxxxI will be added that will be issued only when there is a change in the installed features. It will only list features that were added.

The current CWWKF0012I message will now list all installed features in every case.

Neither WDT or the maven/gradle tools currently use these informational messages.

---

Documents

When available, add links to required feature documents. Use "N/A" to mark particular documents which are not required by the feature.

• Aha: Externally raised RFE (Aha)

  • N/A

• UFO: Link to Upcoming Feature Overview document

  • Set the Box link to be publicly accessible, with a long expiration (10 years)
  • Click "Share" > select "People with link" > click "Link Settings" > under "Link Experation" select "Disable Shared Link on" > set an experation date ~10 years into the future
  • If you lack permissions, contact OpenLiberty/release-architect

• FTS: Link to Feature Test Summary GH Issue

• Beta Blog: N/A

• GA Blog: https://github.com/OpenLiberty/open-liberty/issues/24985

• ---

  Process Overview

• Prioritization

• Design

• Implementation

• Legal and Translation

• Beta

• GA

  • Focal Point Approvals

• Other Deliverables

General Instructions

The process steps occur roughly in the order as presented. Process steps occasionally overlap.

Each process step has a number of tasks which must be completed or must be marked as not applicable ("N/A").

Unless otherwise indicated, the tasks are the responsibility of the Feature Owner or a Delegate of the Feature Owner.

If you need assistance, reach out to the OpenLiberty/release-architect.

Important: Labels are used to trigger particular steps and must be added as indicated.

---

Prioritization (Complete Before Development Starts)

The (OpenLiberty/chief-architect) and area leads are responsible for prioritizing the features and determining which features are being actively worked on.

Prioritization

• [x] Feature added to the "New" column of the Open Liberty project board

  • Epics can be added to the board in one of two ways:
  • From this issue, use the "Projects" section to select the appropriate project board.
  • From the appropriate project board click "Add card" and select your Feature Epic issue

• [x] Priority assigned

  • Attend the Liberty Backlog Prioritization meeting

• ---

  Design (Complete Before Development Starts)

Design preliminaries determine whether a formal design, which will be provided by an Upcoming Feature Overview (UFO) document, must be created and reviewed. A formal design is required if the feature requires any of the following: UI, Serviceability, SVT, Performance testing, or non-trivial documentation/ID.

Design Preliminaries

• [x] UI requirements identified. (Owner and UI focal point)
• [x] ID requirements identified. (Owner and ID focal point)
  • Refer to Documenting Open Liberty.
  • Feature Owner adds label ID Required, if non-trivial documentation needs to be created by the ID team.
  • ID adds label ID Required - Trivial, if no design will be performed and only trivial ID updates are needed.
• [x] Serviceability Requirements Identified. (Owner and Serviceability focal point)
• [x] SVT Requirements Identified. (Owner and SVT focal point)
• [x] Performance testing requirements identified. (Owner and Performance focal point)

Design

• [ ] POC Design / UFO review requested.
  • Owner adds label Design Review Request
• [ ] POC Design / UFO review scheduled.
  • Follow the instructions in POC-Forum repo
• [ ] POC Design / UFO review completed.
• [ ] POC / UFO Review follow-ons completed.
• [ ] Design / UFO approved. (OpenLiberty/chief-architect) or N/A
  • (OpenLiberty/chief-architect) adds label Design Approved
  • Add the public link to the UFO in Box to the Documents section.
  • The UFO must always accurately reflect the final implementation of the feature. Any changes must be first approved. Afterwards, update the UFO by creating a copy of the original approved slide(s) at the end of the deck and prepend "OLD" to the title(s). A single updated copy of the slide(s) should take the original's place, and have its title(s) prepended with "UPDATED".

No Design

• [x] No Design requested.
  • Owner adds label No Design Approval Request
• [x] No Design / No UFO approved. (OpenLiberty/chief-architect) or N/A
  • Approver adds label No Design Approved

FAT Documentation

• [x] "Feature Test Summary" child task created

  • Use the Feature Test Summary Template
  • Add FTS issue link to the Documents section.

• ---

  Implementation

A feature must be prioritized before any implementation work may begin to be delivered (inaccessible/no-ship). However, a design focused approach should still be applied to features, and developers should think about the feature design prior to writing and delivering any code.
Besides being prioritized, a feature must also be socialized (or No Design Approved) before any beta code may be delivered. All new Liberty content must be inaccessible in our GA releases until it is Feature Complete by either marking it kind=noship or beta fencing it.
Code may not GA until this feature has obtained the "Design Approved" or "No Design Approved" label, along with all other tasks outlined in the GA section.

Feature Development Begins

• [x] Add the In Progress label

Legal and Translation

In order to avoid last minute blockers and significant disruptions to the feature, the legal items need to be done as early in the feature process as possible, either in design or as early into the development as possible. Similarly, translation is to be done concurrently with development. Both MUST be completed before Beta or GA is requested.

Legal (Complete before Feature Complete Date)

• [x] Changed or new open source libraries are cleared and approved, or N/A. (Legal Release Services/Cass Tucker/Release PM).
• [x] Licenses and Certificates of Originality (COOs) are updated, or N/A

Translation (Complete 1 week before Feature Complete Date)

• [x] PII updates are merged, or N/A. Note timing with translation shipments.

• ---

  Beta

In order to facilitate early feedback from users, all new features and functionality should first be released as part of a beta release.

Beta Code

• [ ] Beta fence the functionality
  • kind=beta, ibm:beta, ProductInfo.getBetaEdition()
• [ ] Beta development complete and feature ready for inclusion in a beta release
  • Add label target:beta and the appropriate target:YY00X-beta (where YY00X is the targeted beta version).
• [ ] Feature delivered into beta
  • (OpenLiberty/release-manager) adds label release:YY00X-beta (where YY00X is the first beta version that included the functionality).

Beta Blog (Complete 1.5 weeks before beta eGA)

• [ ] Beta blog issue created and populated using the Open Liberty BETA blog post template.

  • Add a link to the beta blog issue in the Documents section.

• ---

  GA

A feature is ready to GA after it is Feature Complete and has obtained all necessary Focal Point Approvals.

Feature Complete

• [x] Feature implementation and tests completed.
  • [x] All PRs are merged.
  • [x] All epic and child issues are closed.
  • [ ] All stop ship issues are completed.
• [ ] Legal: all necessary approvals granted.
• [x] Translation: All messages translated or sent for translation for upcoming release
• [x] GA development complete and feature ready for inclusion in a GA release
  • Add label target:ga and the appropriate target:YY00X (where YY00X is the targeted GA version).
  • Inclusion in a release requires the completion of all Focal Point Approvals.

Focal Point Approvals (Complete by Feature Complete Date)

These occur only after GA of this feature is requested (by adding a target:ga label). GA of this feature may not occur until all approvals are obtained.

All Features

• [ ] APIs/Externals Externals have been reviewed or N/A. (OpenLiberty/externals-approvers)
  • Approver adds label focalApproved:externals
• [ ] Demo Demo is scheduled for an upcoming EOI or N/A. (OpenLiberty/demo-approvers)
  • Approver adds label focalApproved:demo.
• [ ] FAT All Tests complete and running successfully in SOE or N/A. (OpenLiberty/fat-approvers)
  • Approver adds label focalApproved:fat.
• [ ] Globalization Translation and TVT are complete or N/A. (OpenLiberty/globalization-approvers)
  • Approver adds label focalApproved:globalization.

Design Approved Features

• [ ] Accessibility Accessibility testing completed or N/A. (OpenLiberty/accessibility-approvers)
  • Approver adds label focalApproved:accessibility.
• [ ] ID Documentation is complete or N/A. (OpenLiberty/id-approvers)
  • Approver adds label focalApproved:id.

  • NOTE: If only trivial documentation changes are required, you may reach out to the ID Feature Focal to request a ID Required - Trivial label. Unlike features with regular ID requirement, those with ID Required - Trivial label do not have a hard requirement for a Design/UFO.

• [ ] Performance Performance testing is complete or N/A. (OpenLiberty/performance-approvers)
  • Approver adds label focalApproved:performance.
• [ ] Serviceability Serviceability has been addressed or N/A. (OpenLiberty/serviceability-approvers)
  • Approver adds label focalApproved:sve.
• [ ] STE Skills Transfer Education chart deck is complete or N/A. (OpenLiberty/ste-approvers)
  • Approver adds label focalApproved:ste.
• [ ] SVT System Verification Test is complete or N/A. (OpenLiberty/svt-approvers)
  • Approver adds label focalApproved:svt.

Remove Beta Fencing (Complete by Feature Complete Date)

• [ ] Beta guards are removed, or N/A
  • Only after all necessary Focal Point Approvals have been granted.

GA Blog (Complete by Feature Complete Date)

• [ ] GA Blog issue created and populated using the Open Liberty GA release blog post template.
  • Add a link to the GA Blog issue in the Documents section.

Post GA

• [ ] Replace target:YY00X label with the appropriate release:YY00X. (OpenLiberty/release-manager)

• ---

  Other Deliverables

• [ ] Standalone Feature Blog Post A blog post specifically about your feature or N/A. (OpenLiberty/release-architect)

  • This should be strongly considered for larger or more prominent features.
  • Follow instructions in the blogs repo.

• [ ] OL Guides OL Guides assessment is complete or N/A. (OpenLiberty/guide-assessment)

• [ ] Dev Experience Developer Experience & Tools work is complete or N/A. (OpenLiberty/dev-experience-assessment)

• ---