Auto-generation of features in server configuration

Determine suggested Liberty features based on the user’s application and enable them in the project’s server configuration during development.

This can be done as part of the build plugins/dev mode.

Note: The following additional scenarios are mostly out of scope (see expected behaviour on slide 18 of UFO) but to be determined as possible future enhancements/features:

Alternative runtime implementations
Basic Extensions using Liberty Libraries (bells) feature
Spring Boot features
Features that require extra configuration

When ready, add links to the Upcoming Feature Overview document and Feature Test Summary issue:

UFO: https://ibm.box.com/s/fbly7w2zqlcqb2coto1aj2qy57l6as20
FTS:
List of Steps to complete or get approvals / sign-offs for Onboarding to the Liberty release (GM date)

Instructions:

Do the actions below and mark them complete in the checklist when they are done.
Make sure all feature readiness approvers put the appropriate tag on the epic to indicate their approval.

Design

Before Development Starts or 8 weeks before Onboarding

[x] POC Design / UFO Review Scheduled (David Chang) or N/A.
[x] POC Design / UFO Reviewed (Feature Owner) or N/A.
[x] Complete any follow-ons from the POC Review.
[ ] Design / UFO Approval (Alasdair Nottingham) or N/A.
[ ] No Design / No UFO Approval (Arthur De Magalhaes - cloud / Alasdair Nottingham - server) or N/A.
[x] SVT Requirements identified. (Epic owner / Feature owner with SVT focal point)
[x] ID Requirements identified. (Epic owner / Feature owner with ID focal point)
[ ] Create a child task of this epic entitled "FAT Approval Test Summary". Add the link in above.

Legal

3 weeks before Onboarding

[ ] Identify all open source libraries that are changing or are new. Work with Legal Release Services (Cass Tucker or Release PM) to get open source cleared and approved. Or N/A. (Epic Owner). New or changed open source impacts license and Certificate of Originality.

Translation

3 weeks before Onboarding

[ ] All new or changed PII messages are checked into the integration branch, before the last translation shipment out. (Epic Owner)

Feature Complete

2 weeks before Onboarding

[x] Implementation complete. (Epic owner / Feature owner)
[ ] All function tests complete. Ready for FAT Approval. (Epic owner / Feature owner)
[x] Review all known issues for Stop Ship. (Epic owner / Feature owner / PM)

Focal Point Approvals

2 to 1 week before Onboarding

You MUST have the Design Approved or No Design Approved label before requesting focal point approvals.

All features (both "Design Approved" and "No Design Approved")

[ ] FAT - (Kevin Smith). SOE FATS are running successfully or N/A . Approver adds label focalApproved:fat to the Epic in Github.
[ ] Demo - (Tom Evans or Chuck Bridgham). Demo is scheduled for an upcoming EOI. Approver adds label focalApproved:demo to the Epic in Github.
[ ] Globalization (Sam Wong - Liberty / Simy Cheeran - tWAS). Translation is complete or N/A. TVT - complete or N/A. Approver adds label focalApproved:globalization to the Epic in Github.

"Design Approved" features

[ ] Accessibility - (G Scott Johnston). Accessibility testing is complete or N/A. Approver adds label focalApproved:accessibility to the Epic in Github.
[ ] ID - (Kareen Deen). Documentation work is complete or N/A . Approver adds label focalApproved:id to the Epic in Github.
[ ] Performance - (Jared Anderson). Performance testing is complete with no high severity defects or N/A . Approver adds label focalApproved:performance to the Epic in Github.
[ ] Serviceability - (Don Bourne). Serviceability has been addressed.
[ ] STE - (Swati Kasundra). STE chart deck is complete or N/A . Approver adds label focalApproved:ste to the Epic in Github.
[ ] SVT - (Brian Hanczaryk- APS). SVT is complete or N/A . Approver adds label focalApproved:svt to the Epic in Github.

Ready for GA

1 week before Onboarding

[ ] No Stop Ship issues for the feature. (Epic owner / Feature owner / Release PM)
[ ] Ship Readiness Review and Release Notes completed (Epic owner / Feature owner / Release PM)
[ ] Github Epic and Epic's issues are closed / complete. All PRs are committed to the master branch. (Epic owner / Feature owner / Backlog Subtribe PM)

Other deliverbles

[ ] OL Guides - (Yee-Kang Chang). Assessment for OL Guides is complete or N/A.
[ ] WDT - (Leonard Theivendra). WDT work complete or N/A.
[ ] Blog - (Laura Cowen) Blog article writeup (Epic owner / Feature owner / Laura Cowen)

Comments from today's UFO socialization are below. A part 2 will be required to complete the socialization. We ended on the Feature Design (special features scenarios) slide

Technical Background

Server configuration definition should also include config dropins

Feature Design (High Level Flow Chart)

Need to document how it behaves on first launch, not just when files change. On first launch everything is passed in.

Feature Design (Deleting features)

Look into having a message for API features listed in server.xml configuration that are not needed when looking at the binary scanner analysis.

Feature Design (Binary Scanner downloads)

Add a statement that says that when dev mode is used with a new version of the plugins with this support that it will do this by default.

Feature Design (different versions of MP, Java EE, Jakarta EE)

May need to revisit to include build dependencies in helping to determine which version of MP / JavaEE / JakartaEE version to use.

Feature Design (special feature scenarios)

JDBC special case? Are we making the right choice? Do we take the Java version into consideration? If using Java 8 default to jdbc 4.2. If Java 11 or higher default to jdbc 4.3 for example.
Make sure there is a test for the server.xml having a *Container feature that we do not add the non Container version of the generated feature list.
Security RolesAllowed being defined does not necessarily mean that the user wants to have security enabled. Need to make sure that is tested and determine if we have the behavior that we want for that scenario.

Comments from today's UFO socialization part 2:

Java APIs/SPIs

Should the method name be different than driveScanMavenFeatureList? It is used by by both Maven and Gradle plugin. Should the word Maven be removed?
Should the package name be changed from com.ibm.ws to differentiate it as an API / externalized function. With open liberty we use the io.openliberty package prefix for APIs now.
There is a mix of array and Collection. We should just use Collection APIs. String[] should change to List instead.
currentFeatures should be a Set instead
Combine targetJavaEE and targetJakarataEE to one.
Constants for the Strings in a public class or an enum of the target fields would be good to have.

Admin / Config / Command Line

generateFeatures also available as plugin config. Add a slide about how to configure generateFeatures using plugin config. We talked about it being something we planned for maven, but not for gradle yet.

Migration Impact

Add a statement about a dependency on a new version of binary scanner to be able to ship this feature

Technical Background

Add LMP and LGP abbreviations so that people know what they mean throughout the UFO

Generic comments

Binary Scanner should not be capitalized since it isn't the official name of the product.
Look to have an internal way to specify a specific version of the binary scanner in case moving to new versions causes an issue.

Update for slide 24: The standalone Gradle task to generate features will be called generateFeatures instead of libertyGenerateFeatures for simplicity and to more closely align with the existing task of installFeature.

Updates to the UFO from suggestions:

Technical Background

Server configuration definition should also include config dropins
- Updated on slide 5

Feature Design (High Level Flow Chart)

Need to document how it behaves on first launch, not just when files change. On first launch everything is passed in.
- Updated in flow chart on slide 13

Feature Design (Binary Scanner downloads)

Add a statement that says that when dev mode is used with a new version of the plugins with this support that it will do this by default.
- Added to slide 15

Feature Design (different versions of MP, Java EE, Jakarta EE)

May need to revisit to include build dependencies in helping to determine which version of MP / JavaEE / JakartaEE version to use.
- Updated on slides 17 (Feature Design) and 20 (Java APIs/SPIs)

Java APIs/SPIs

The API that is implemented (shown on slide 20) has been updated to address the many suggestions

Admin / Config / Command Line

generateFeatures also available as plugin config. Add a slide about how to configure generateFeatures using plugin config. We talked about it being something we planned for maven, but not for gradle yet.
- Added slide 23

Migration Impact

Add a statement about a dependency on a new version of binary scanner to be able to ship this feature
- Added to slide 38

Technical Background

Add LMP and LGP abbreviations so that people know what they mean throughout the UFO
- Updated on slide 5

Generic comments

Binary Scanner should not be capitalized since it isn't the official name of the product.
- Updated throughout

Other updates to the UFO:

Updated the flow chart on slide 13 (Feature Design - High Level Flow Chart) to show that with implementation we now perform the "optimize" generate features operation on server configuration and build file changes (previously, we had planned to perform the "incremental" generate features operation)
Added a couple bullets on slide 14 (Feature Design) to note the new scenarios we are performing the "optimize" generate features operation
Added new user facing error message to slide 36 (Serviceability)

Issues opened to address other suggestions:

Feature Design (Deleting features)

Look into having a message for API features listed in server.xml configuration that are not needed when looking at the binary scanner analysis.
- Issue for this here: https://github.com/OpenLiberty/ci.maven/issues/1377

Feature Design (special feature scenarios)

JDBC special case? Are we making the right choice? Do we take the Java version into consideration? If using Java 8 default to jdbc 4.2. If Java 11 or higher default to jdbc 4.3 for example.
- Issue to address this scenario: https://github.ibm.com/was-migration-tools/wamt/issues/4984
Make sure there is a test for the server.xml having a *Container feature that we do not add the non Container version of the generated feature list.
- Issue for this here: https://github.ibm.com/was-migration-tools/wamt/issues/4985
Security RolesAllowed being defined does not necessarily mean that the user wants to have security enabled. Need to make sure that is tested and determine if we have the behavior that we want for that scenario.
- Issue opened here: https://github.com/OpenLiberty/ci.maven/issues/1473
- This issue is for generally allowing users to remove any features that they do not want generated. The plan is to only address this if users raise it as a problem.

Comment on slide 11:

It says the generated features xml will be written into the source folder, but this is very atypical of a build plugin. I can guess what the user story for this capability would be, but then I'd expect that to be called out on slide 8. I think solving that user story would be useful, but I wouldn't expect it to be on by default as this implies. I think the general principle is that maven builds do not change the content in the source folders.
I'm still worrying about the license the binary scanner dependency is released under. That isn't called out here, but I'm concerned about the possibility that it not be compatible with EPL.

OpenLiberty / open-liberty