Closed Emily-Jiang closed 3 years ago
Actions/Followups from first UFO review
Actions/Followups from first UFO review
@NottyCode Added "Design Approval Request". Thanks.
You've requested focal point approvals. For globalization approvals, 21001/2 translations are not back yet. Can you clarify whether there are any translations & I can approve now or whether it was sent & returned previously? Thanks.
@samwatibm Hi Sam. I believe the nlprops or translation content was delivered in 20.0.0.12 or before for this feature.
Serviceability Approval Comment - Please answer the following questions for serviceability approval:
WAD -- does the WAD identify the most likely problems customers will see and identify how the feature will enable them to diagnose and solve those problems without resorting to raising a PMR? Have these issues been addressed in the implementation? Yes - The WAD notes messages that have been added to cover new errors that are due to the new config attributes as well as mis-matches in the token and the server configs.
Test and Demo -- As part of the serviceability process we're asking feature teams to test and analyze common problem paths for serviceability and demo those problem paths to someone not involved in the development of the feature (eg. L2, test team, or another development team).
a) What problem paths were tested and demonstrated?
SVT -- SVT team is often the first team to try new features and often encounters problems setting up and using them. Note that we're not expecting SVT to do full serviceability testing -- just to sign-off on the serviceability of the problem paths they encountered. (refer to comment from hanczaryk - https://github.com/OpenLiberty/open-liberty/issues/11022#issuecomment-765396344) a) Who conducted SVT tests for this feature? b) Do they agree that the serviceability of the problems they encountered is sufficient to avoid PMRs, or that L2 should be able to quickly address those problems without need to engage L3?
Which L2 / L3 queues will handle PMRs for this feature? Ensure they are present in the contact reference file and in the queue contact summary, and that the respective L2/L3 teams know they are supporting it. Ask Don Bourne if you need links or more info.
Does this feature add any new metrics or emit any new JSON events? If yes, have you updated the JMX metrics reference list / Metrics reference list / JSON log events reference list in the Open Liberty docs?
I've started an initial FAT review, it looks promising so far but there are a number of known that unresolved issues with the tests that I see you're aware of.
L2 has requested STE slides for this feature. The STE template can be found at the links below. You can use either one to create the education.
Slide Template: https://ibm.box.com/s/1an42g7zdgmaj84w7dft0indqfgi8ffm
Github Template: https://pages.github.ibm.com/WASL3/site/STE/about
Please upload the completed slides to the same 'STE Archive' BOX folder or provide me the Github link. Thanks!
@teddyjtorres , please add comment with answers for serviceability approval (even if we don't have SVT complete yet it's helpful for seeing if everything else is ready for this approval)
Demo scheduled for February 2nd.
@steven1046 There is no Accessibility needed for this Epic. Please let me know if Serviceability approval can be resolved as "N/A", thanks.
@tevans78 or @cbridgha The demo was scheduled for the February 2nd's EOI. Please let me know if demo focal approval can be resolved. Thanks.
@Emily-Jiang @chirp1 The ID issue is https://github.com/OpenLiberty/docs/issues/3511.
Here are the SVT answers for https://github.com/OpenLiberty/open-liberty/issues/11022#issuecomment-744497280
a) Who conducted SVT tests for this feature?
Brian Hanczaryk
b) Do they agree that the serviceability of the problems they encountered is sufficient to avoid PMRs, or that L2 should be able to quickly address those problems without need to engage L3? Yes, I agree that the serviceability of any problems encountered is sufficient to avoid PMRs, or that L2 should be able to quickly address those problems without the need to engage L3.
@donbourne
WAD -- does the WAD identify the most likely problems customers will see and identify how the feature will enable them to diagnose and solve those problems without resorting to raising a PMR? Have these issues been addressed in the implementation? A: Yes
Test and Demo -- As part of the serviceability process we're asking feature teams to test and analyze common problem paths for serviceability and demo those problem paths to someone not involved in the development of the feature (eg. L2, test team, or another development team).
a) What problem paths were tested and demonstrated?
b) Who did you demo to?
c) Do the people you demo'd to agree that the serviceability of the demonstrated problem scenarios is sufficient to avoid PMRs for any problems customers are likely to encounter, or that L2 should be able to quickly address those problems without need to engage L3?
A: Demo scheduled for EOI.
SVT Answered question 3.
Which L2 / L3 queues will handle PMRs for this feature? Ensure they are present in the contact reference file and in the queue contact summary, and that the respective L2/L3 teams know they are supporting it. Ask Don Bourne if you need links or more info. A: WAS: L2 SEC Team and WAS L3: Security SSO
Does this feature add any new metrics or emit any new JSON events? If yes, have you updated the JMX metrics reference list / Metrics reference list / JSON log events reference list in the Open Liberty docs? A: N/A. The feature does not add any new metrics or new JSON events.
@mhldr The TCK issues were resolved in https://github.com/OpenLiberty/open-liberty/pull/15606. We're ready for FAT approval. Thanks.
@teddyjtorres , demo (from question 2) needs to explicitly cover showing the problem scenarios so that participants can comment on whether they think the serviceability of the demonstrated problems is sufficient. EOI demos often just cover the "green thread" path, so need to ensure you're demoing the problem paths too if that's where you're planning to get serviceability feedback.
@skasund I've uploaded the STE slides for this feature to the STE Archive Box folder: https://ibm.box.com/s/lp6kczjtib15limcnb6krm2x3abw2xm0
@ayoho Thanks for the STE slides
@donbourne We will demo also to our L3, who has not been involved in the development or testing of the feature.
Our FAT tester also covered testing the messages.
@mhldr Please let us know if there is anything else needed for FAT approval. Thanks.
@donbourne We will demo also to our L3, who has not been involved in the development or testing of the feature.
That sounds good - please update when you have completed the demo and indicate if the L3 team is happy with serviceability capabilities so I can sign off. I'm ok if minor serviceability issues are found as long as you show that you have opened issues to address them in the next release.
@donbourne we have completed the demo with our L3 @barbj. She is happy with what was tested as well as messages/config descriptions with a few minor updates (issue: #15755). Let me know if you need anything else.
Based on my conversation with MP JWT, MP JWT 1.2 will be released part of MP 4.0 Support the next MicroProfile JWT release 1.2 and baseline Jakarta EE8.
UFO: https://ibm.box.com/s/fi8gp4aag4t7goza3mq18umf6l2cv5tw Feature Test Summary (FTS): https://github.com/openliberty/open-liberty/issues/14978
Stories under this feature:
Doc updates:
Beta Blog Post:
GA Blog Post:
OL Guide updates:
[ ] https://github.com/OpenLiberty/guides-common/issues/525
List of Steps to complete or get approvals / sign-offs for Onboarding to the Liberty release (GM date)
Instructions:
TARGET COMPLETION DATE Before Development Starts or 8 weeks before Onboarding
[x] POC Design / UFO Review Scheduled (David Chang) or N/A.
[x] POC Design / UFO Reviewed (Feature Owner) or N/A.
[x] Complete any follow-ons from the POC Review.
[x] Add "Design Approval Request" tag
[x] Design / UFO Approval (Alasdair Nottingham) or N/A.
[x] No Design / No UFO Approval (Arthur De Magalhaes - cloud / Alasdair Nottingham - server) or N/A.
[x] SVT Requirements identified. (Epic owner / Feature owner with SVT focal point)
[x] ID Requirements identified. (Epic owner / Feature owner with ID focal point)
[x] Create a child task of the epic entitled "FAT Approval Test Summary". Add and fill in the template as described here: https://github.ibm.com/was-liberty/WS-CD-Open/wiki/Feature-Review-(Feature-Test-Summary-Process . Mark "FAT complete" on the Epic when it's ready for review
TARGET COMPLETION DATE 3 weeks before Onboarding
[x] Identify all open source libraries that are changing or are new. Work with Legal Release Services (Cass Tucker or Release PM) to get open source cleared and approved. Or N/A. (Epic Owner). New or changed open source impacts license and Certificate of Originality.
TARGET COMPLETION DATE 3 weeks before Onboarding
[x] All new or changed PII messages are checked into the integration branch, before the last translation shipment out. (Epic Owner)
TARGET COMPLETION DATE 2 weeks before Onboarding
[x] Implementation complete. (Epic owner / Feature owner)
[x] All function tests complete. Ready for FAT Approval. (Epic owner / Feature owner)
[ ] Review all known issues for Stop Ship. (Epic owner / Feature owner / PM)
APPROVALS with TARGET COMPLETION DATE 2 to 1 week before Onboarding
Prerequisites
You must have the Design Approved or No Design Approved label on the GitHub Epic.
If the feature is not part of Liberty, set
target:ga
label in this epic so it is on the approvers' radar.Approvals
[x] Accessibility - (Steven Zvonek). Accessibility testing is complete or N/A. Approver adds label focalApproved:accessibility to the Epic in Github.
[x] FAT Liberty SOE - (Martin Holder). SOE FATS are running successfully or N/A . Approver adds label focalApproved:fat to the Epic in Github.
[x] Globalization (Sam Wong - Liberty / Simy Cheeran - tWAS). Translation is complete or N/A. TVT - complete or N/A. Approver adds label focalApproved:globalization to the Epic in Github.
[x] ID - (Kareen Deen). Documentation work is complete or N/A . Approver adds label focalApproved:id to the Epic in Github.
[x] Performance - (Jared Anderson). Performance testing is complete with no high severity defects or N/A . Approver adds label focalApproved:performance to the Epic in Github.
[ ] Serviceability - (Don Bourne). Serviceability has been addressed.
[ ] STE - (Swati Kasundra). STE chart deck is complete or N/A . Approver adds label focalApproved:ste to the Epic in Github.
[x] SVT - (Greg Ecock - Cloud, Brian Hanczaryk- APS). SVT is complete or N/A . Approver adds label focalApproved:svt to the Epic in Github.
[x] Demo - (Liberty only - Tom Evans or Chuck Bridgham). Demo is scheduled for an upcoming EOI. Approver adds label focalApproved:demo to the Epic in Github.
TARGET COMPLETION DATE 1 week before Onboarding
[ ] No Stop Ship issues for the feature. (Epic owner / Feature owner / Release PM)
[ ] Ship Readiness Review and Release Notes completed (Epic owner / Feature owner / Release PM)
[ ] Github Epic and Epic's issues are closed / complete. All PRs are committed to the master branch. (Epic owner / Feature owner / Backlog Subtribe PM)
NOT REQUIRED FOR A FEATURE
[ ] OL Guides - (Yee-Kang Chang). Assessment for OL Guides is complete or N/A.
[ ] WDT - (Erin Harris). WDT work complete or N/A.
Related Deliverables TARGET COMPLETION DATE General Availability
[ ] Blog article writeup (Epic owner / Feature owner / Laura Cowen)