armstro commented 4 years ago

Proposal is to create a set of conformance criteria to be a Zowe Support Provider.

Examples

Agreement of what is included in a "core" Zowe distribution and acceptance of Zowe/OMP project policies since it will be shared support across multiple possible vendors
Agreement to recognize the Genuine Zowe process (hash tag/ watermark,...)
Agreement to distribute core Zowe as a separate upgradeable/serviceable component (SMP/E FMID, its own PSI package, etc.)
Use of Zowe build process and regression testing for core Zowe
Use of Zowe diag tools for core
Timely reporting of any security vulnerabilities to the Zowe community using the discrete reporting process
Willingness to assist in security exposures in code/repos where they are committers

jmertic commented 4 years ago

Whenever y'all are ready to start here, let me know.

armstro commented 4 years ago

John - my crystal ball is foggy on this but my guess is this is 30 days out.....TSC mission and establishment and joint PI Planning are higher priorities at the moment.

jmertic commented 3 years ago

Want to bump this up - starting to see more public movement and advise that getting ahead of more growth would be something to consider sooner than later.

armstro commented 3 years ago

Once we get TSC moving we will come back to this item ......

jmertic commented 3 years ago

OK

armstro commented 3 years ago

OK - I wanted to dust off this item and see about getting it started..... @jmertic you mentioned Kubernetes has a program for Support - I found this https://www.cncf.io/certification/kcsp/ - do you have any other info? The good news is the criteria is short and sweet but the problem seems to be it is difficult to apply to Zowe....(like Three or more engineers who pass the Certified Kubernetes Administrator (CKA) exam. )..... but the list is a place to start. Do you have other examples for Support Conformance?

jmertic commented 3 years ago

That's probably the best I've seen thus far.

I'd not get hung up on the precise details, but more think about what you would hope that anyone saying they could support Zowe should be able to do. This might be having certain skillsets on staff. This might be community involvement. Might have something related other capacities. I'd encourage y'all to be creative on the way you think about this.

armstro commented 3 years ago

Update - Peter, Mike D and myself have been working on Support Conformance criteria in a couple documents (modeled after the current Zowe conformance programs for Web UI, API ML, etc.) - summary of status and key issues here - more work to do:

Support conformance would be subdivided like the other conformance programs meaning a vendor could choose to only offer Support on part of Zowe (CLI and Explorer for example) or all of Zowe (meaning multiple badges but when is too many badges too confusing)
Since we do not have exams like Kubernetes does - there is discussion on what "credentials" are needed by a Service Provider - at a minimum they need at least contributers (or committers) on the squad(s) where they claim Support (like CLI/Explorer)
There is a question on what role the squad plays in reporting status of contributor participation/competence - if a badged Support provider contributor does not show adequate skills or does not keep up with squad activities what should be done? We want to avoid a vendor just reporting issues and not contributing to working solutions
Can any OMP member be a Support Provider or is there an additional membership to be considered? (How to avoid Support being offered only by big firms?)
The Support "scope" generally relates to "core" Zowe capabilities (for example not incubators - although any vendor could offer incubator assistance for a fee but that is not "support") - do we need to revisit what core means and who/how determines core?
What is OMP role if a Support Provider has complaints from customers? Under what circumstances is a badge revoked?

Plans are for additional working sessions and to include others with experience with the existing conformance programs to gain lessons learned and advice.

jmertic commented 3 years ago

Glad this is taking shape! A few thoughts based on the questions...

Requiring squad participation is probably the best bet right now, until some sort of 'Zowe' coursework could go live ( nudge nudge ;-) ). One thought to think about is that you likely will have vendors interested in support conformance that might not have enough of a technical bench to contribute upstream - not to say that squad participation shouldn't be important criteria, but something to think about.
I would imaging running the support provider program fee requirements like we do for other Zowe Conformance ( free for members, cost for non-members ) would make the most sense.
Although we've been fortunate to not have challenges on conformance yet, the way I would see it work is to first attempt to mediate the concern, and if that is unsuccessful look to remove the provider from the program.

Please pull me in when appropriate.

armstro commented 3 years ago

Many thanks to Rose for keeping up moving on the Support Conformance Program. Latest draft is attached - comments:

1) We state Support can be Comprehensive (meaning all of Core Zowe) or a subset (i.e, Partial). We also state "Best Practices" implies Support beyond Core - do we want to coin a term other than Best Practice - like Comprehensive+ ?

2) Do we need to a "finger print" program equivalent for non-z/OS components ? If one already exists does it need to be made more visible to the community?

3) Question on "no forked code" - does it just need to state abide by EPL2.0 with any code changes?

4) Question on timeliness of reporting security vulnerabilities - I think 3-5 days? What would hold up a report being made? Do we need to say vulnerabilities would not be made public until a mitigation is ready for distribution?

5) I think we need to work with the TSC on the definition of Core Zowe. I think TSC owns the declaration of core changes when a component, app, extension is ready to be considered core. Support vendors not ready to support a new core component would have the ability to work with the TSC.

Zowe Support Conformance V1.xlsx

jmertic commented 3 years ago

This does look good - thanks Rose!

Some comments...

Support Vendor agrees to support any binary-equivalent of all Zowe core components, regardless of where obtained

This feels like it could get messy, and considering some of the recent activities we've seen with ransomware the phrase "regardless of where obtained" is also something that should be discouraged.

We should also define "binary-equivalent"

Support Vendor Agrees: to abide by "no forked code distributions of Zowe" - for either full distributions of Zowe OR any partial code distributed as a core Zowe enhancement or fix

Something that should be defined, as Rose pointed out.

Support Vendor confirms: comprehensive penetration tests are conducted annually by said Support Vendor and said Support Vendor is able to provide to OMP, reports demonstrating as such, when asked to do so

What does this mean?

Support Vendor agrees: to adhere to all documented Zowe TSC Engineering Standards [include a link to the standards here] for any support-related code change (fixes, patches, PTFs etc.) submission

Support Vendor Confirms: Said Vendor has read, understands and agrees to abide by the Zowe TSC Support-Vendor-Fix Process [include a link to the process here]

Support Vendor agrees: to adhere to all documented Zowe TSC Documentation Standards [include a link to the standards here] for any support-related code change (fixes, patches, PTFs etc.) submission that includes documentation

I'd think this would basically be the contribution guidelines, right? Not sure if there is some difference that would be expected here.

Support Vendor Agrees: to contribute all fixes to the Zowe community in a timely manner, as soon as possible and such contributions may be delayed for no longer than 3 months (90 days)

The last part seems to be putting a contract on the squads, which should have the right to accept or reject the contribution ( or even delay it if it makes sense to apply at a later time ).

"Active contributors" should also be defined, but I'm also wondering if that could exclude those who don't have the right skills on the bench to contribute code.

armstro commented 3 years ago

John - if you can join the ZLC call today maybe we can talk thru some of these

Regarding multiple sources of the code - we already have zowe.org, 2 vendor sites, npmjs registry, Docker hub (not yet core) as multiple sources .....I don't think we can control where the code is hosted (don't want to limit other Support Providers from having their own distribution)
Regarding binary equivalent - so we have focused on ways to validate the code is genuine via "finger print app" (fancy name for hash) shipped in the z/OS components and (we think) the VS Code and npmjs registries have something similar.
See my comments regarding forked code - I think we just wanting to say they agree to EPL2.0......Support Vendors may need to fork the code for an emergency fix but need to agree to contribute the fix back to the community.
I think the intent is the vendor should participate in the Security subgroup that works on and security vulnerability that are found (it does not need to be limited to pen testing but any testing or defect report)
Adhering the TSC policies does need to be clarified - We need to discuss with the TSC - they have not seen this yet. - we just mean the Support Provider needs to follow the TSC policies - using automated testing, documentation, etc.
We can discuss "active contributor" definition - the point is that a Support person/org should do more than raise issues for the squad to address and we would prefer a contributor/committer. How we define this idea of "active" is something to discuss.

Bruce Armstrong IBM Z Product Manager assigned to zowe.org 4205 S MIAMI BLVD, DURHAM NC 27703-9141 Email: @.*** Tel: 919-254-8773 Cell: 919-931-3132

From: John Mertic @.> To: zowe/zlc @.> Cc: Bruce Armstrong @.>, Assign @.> Date: 05/25/2021 07:33 AM Subject: [EXTERNAL] Re: [zowe/zlc] Establishment of Zowe Support Provider Conformance Program (#182)

This does look good - thanks Rose! Some comments... Support Vendor agrees to support any binary-equivalent of all Zowe core components, regardless of where obtained This feels like it could get messy, and considering some of the recent activities This does look good - thanks Rose! Some comments... Support Vendor agrees to support any binary-equivalent of all Zowe core components, regardless of where obtained This feels like it could get messy, and considering some of the recent activities we've seen with ransomware the phrase "regardless of where obtained" is also something that should be discouraged. We should also define "binary-equivalent" Support Vendor Agrees: to abide by "no forked code distributions of Zowe"

for either full distributions of Zowe OR any partial code distributed as a core Zowe enhancement or fix Something that should be defined, as Rose pointed out. Support Vendor confirms: comprehensive penetration tests are conducted annually by said Support Vendor and said Support Vendor is able to provide to OMP, reports demonstrating as such, when asked to do so What does this mean? Support Vendor agrees: to adhere to all documented Zowe TSC Engineering Standards [include a link to the standards here] for any support-related code change (fixes, patches, PTFs etc.) submission Support Vendor Confirms: Said Vendor has read, understands and agrees to abide by the Zowe TSC Support-Vendor-Fix Process [include a link to the process here] Support Vendor agrees: to adhere to all documented Zowe TSC Documentation Standards [include a link to the standards here] for any support-related code change (fixes, patches, PTFs etc.) submission that includes documentation I'd think this would basically be the contribution guidelines, right? Not sure if there is some difference that would be expected here. Support Vendor Agrees: to contribute all fixes to the Zowe community in a timely manner, as soon as possible and such contributions may be delayed for no longer than 3 months (90 days) The last part seems to be putting a contract on the squads, which should have the right to accept or reject the contribution ( or even delay it if it makes sense to apply at a later time ). "Active contributors" should also be defined, but I'm also wondering if that could exclude those who don't have the right skills on the bench to contribute code. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or unsubscribe.

armstro commented 3 years ago

Let me try to summarize the changes made after the ZLC call this morning (and we need to work with the TSC on a few items).

In layman's terms

Used the term authenticated binary to be Supported (and pointer to Zowe doc that describes)
For clarity say support must be all active/maintenance releases in a version
Code changes due to fixes need to be contributed back (consistent with EPL 2.0)
Need to report vulnerabilities (Best practices to work with the Security team)
Use authenticated binary when (if) embedding Zowe
Follow TSC policies and practices
Honor the PTF numbering process
If vendor specific patches are created to inform the community and incorporate back to community (this is a little redundant with the EPL 2.0 comment above and maybe can be struck out)
List the names of contributors/committers

Areas where we need to work with TSC

Process for defining "core"
Confirm there is authentication process for z/OS, distributed, (and containers)
TSC policies and practices (code quality, automated test, doc, etc.) - maybe include practice of 90 day to contribute back and what to do/say about customer validation of a fix
How to define an active contributor?

Zowe Support Conformance V1a - 5-25-21.xlsx

PeterFandelAtRocket commented 3 years ago

I added new text for footnote 1.2 and made items 9 and 11 requirements (x was missing) Zowe.Support.Conformance.V1b.-.5-25-21.xlsx

armstro commented 3 years ago

Zowe.Support.Conformance.V1-Near Final .xlsx OK - I did my best to incorporate all the comments from today's ZLC - please read top to bottom including footnotes. I would like to bring this to a vote ASAP. I think we also agreed TSC will own "what is core".

jmertic commented 3 years ago

Looking better! I think there is some wordsmithing that can be done to clean things up, but nothing major.

PeterFandelAtRocket commented 3 years ago

I read the whole thing and reworded item #2 and also made a bunch of really minor comments.

I am happy to integrate into a new version myself if you want – just say the word Bruce.

From: Bruce Armstrong @.> Sent: Tuesday, June 1, 2021 1:48 PM To: zowe/zlc @.> Cc: Peter Fandel @.>; Assign @.> Subject: Re: [zowe/zlc] Establishment of Zowe Support Provider Conformance Program (#182)

EXTERNAL EMAIL

Zowe.Support.Conformance.V1-Near Final .xlsxhttps://github.com/zowe/zlc/files/6578384/Zowe.Support.Conformance.V1-Near.Final.xlsx OK - I did my best to incorporate all the comments from today's ZLC - please read top to bottom including footnotes. I would like to bring this to a vote ASAP. I think we also agreed TSC will own "what is core".

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/zowe/zlc/issues/182#issuecomment-852325442, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHBGVU5QQXMX5RF7D62CBVTTQUMMZANCNFSM4MSDQJUA.

privacy-policy

This communication and any attachments may contain confidential information of Rocket Software, Inc. All unauthorized use, disclosure or distribution is prohibited. If you are not the intended recipient, please notify Rocket Software immediately and destroy all copies of this communication. Thank you.

armstro commented 3 years ago

@PeterFandelAtRocket happy to have you "hold the pen" on this - I had asked Rose to review but I don't see her on the issue assignee list so she may not be seeing these updates. (and I am not sure she's even in the zlc issues repo to be assigned). My comments to your comments:

Use of Bold was once used to show delta from one version of the spreadsheet to another - bold can be removed
Use up caps and footnote numbering is up to you (Rose might have done the footnotes to be consistent with the other conformance criteria spreadsheets - don't know)
The italics in footnote 3 was to highlight a comment made by Joe regarding 90 day notice of "core changes" (my term) with a fix. We can decide how (if) to include this - I think it makes sense for Support Vendor to notify the community - I don't see how to enforce any of this
The missing links in the spreadsheet is because I don't think the TSC has decided the various policies and documented them yet.

Unless @pdubz3 or Rose have any objections - I say make your changes - leave links as TBD. We make Jakub aware of the "near final" proposal. He's already taken to the TSC the issues we need help with (defining active contributor, confirming there are ways to declare code as "authentic", what is core, etc.) but we need to give the TSC some time to sort that out.

@jmertic we briefly discussed having some "public vetting" of the Support Conformance criteria - if you have particular parties we should seek out for feedback, we can take that next step if you want.

I think we (ZLC) needs to put 1-2 slides together to summarize what we are proposing for easier presentation.

pdubz3 commented 3 years ago

No objection to Peter making the updates he proposed.

pdubz3 commented 3 years ago

Minor wording question, we may have covered this before so I apologize in advance if we have ... if a criterion is indicated as "required" is it less ambiguous to avoid words like "should" in the description in favor of words like "must"? For example in the first section, saying "Customers being supported should not be required to go to a particular vendor distribution channel ..." makes that sound more like a recommendation or best practice. I would suggest "Customers being supported must not be required to go to a particular vendor distribution channel." Thoughts on this?

PeterFandelAtRocket commented 3 years ago

Zowe.Support.Conformance.V1-Near.Final.rev2.xlsx

I applied all my edits as well as replaced "should" with "must" per Mike's previous comment.

I also made the cells containing URLs to be active hyperlinks.

I left the Joe W italicized comment for now.

pdubz3 commented 3 years ago

Thank you @PeterFandelAtRocket. There's a second instance of "should" in one of the footnotes but I'm less certain as to whether it's really a "must". The footnote is about emergency fixes and it reads "Support Vendor Agrees: If any fixes require changes to the Zowe codebase then under the terms of the EPL 2.0 license, the modified code must be made available to the Zowe community. This should be in the form of a git issue be created that describes the original defect together with either a pull request, or a link to where the support vendor has published their updated code. This must be done in a timely manner after the support vendor has made the updated code available to their customer and delayed for no longer than 3 months (90 days). "

PeterFandelAtRocket commented 3 years ago

Replaced other 'should' with 'must' Zowe.Support.Conformance.V1-Near.Final.rev3.xlsx

RASakach commented 3 years ago

@armstro @PeterFandelAtRocket @pdubz3 : Apologies for the late review. This looks really good! I've attached a V4 with some minor comments for your consideration. They do not change the underlying messaging, this was my attempt to make the point a bit more clear. Zowe.Support.Conformance.V1-Near.Final.rev4.xlsx

armstro commented 3 years ago

Thanks Rose - regarding "how the notification" will occur to the TSC regarding emergency fixes - @nkocsis was working with the TSC on how customer issues could be raised and be better visible - maybe there is something that can be done for emergency defects? Just a thought

Regarding filling in some of the links - the TSC is working on how to describe core (https://github.com/zowe/community/pull/1213) , active contributor so we are getting closer to having something to present to John.

nkocsis commented 3 years ago

Once we have a process in place where we know that the squads are reviewing the incoming customer issues then I would expect that it would handle "Emergency" defects.

armstro commented 3 years ago

Proposed 1 slide for July 21 webinar - draft - 2-3 mins max

2021 July Zowe Quarterly Webinar Series - Support Conformance Slide .pptx

pdubz3 commented 3 years ago

Bruce, thanks for taking the time to create a slide for the webinar. As we discussed offline I created a second draft using mostly your content but focusing a little more on the value of the program to the Zowe user, and then the steps a vendor takes and the principles of the program. I left the original but added a second slide for the alternate draft.
2021.July.Zowe.Quarterly.Webinar.Series.-.Support.Conformance.Slide.pptx

armstro commented 3 years ago

Thanks Mike - your slide is great - let's go with it.

armstro commented 3 years ago

Zowe.Support.Conformance.V1-Near.Final.rev5.xlsx

Captured link to define "core" in the spreadsheet

armstro commented 3 years ago

Zowe.Support.Conformance.V1-Near.Near.Final.rev6.xlsx

If we are going to preview (and request comments for) the Support Conformance at the up coming Zowe Quarterly Webinar then I thought I would tweak some of the open gaps we had in prior spreadsheet. Please read closely - I incorporated Rose's comments. I think we need to take to TSC for comment but can be done while we ask for comments in the wider community. Maybe we have a new issue created to link to in the Webinar slides?

Few things to note:

Full time participation in Security team is best practice but I made working part time when a security issue has been identified by the vendor a requirement
Regarding PTF creation - I can't imagine when a vendor would do their own build? I say they need to use the community build process for PTFs but emergency fixes can be done whatever they feel necessary?
I say they need to work with TSC at the next TSC meeting on any code changes needed in core as the result of an emergency fix - emailing someone didn't seem enough (we could have an issue raised with some security tag but I thought meeting might be best way to start)

All of this can be tweaked further

@RASakach for the life of me I can get urls in the text to be actual links - my excel skills are lacking. If you know the steps please advise.

armstro commented 3 years ago

Zowe.Support.Conformance.V1-Final July 19 2021.xlsx

RASakach commented 3 years ago

@armstro Please review to ensure all URLs have been changed - I believe I found all of them. Zowe.Support.Conformance.V1-Final.July.19.2021+links.xlsx

armstro commented 3 years ago

Thanks x100 Rose - links look good and I checked each one - they work!

PeterFandelAtRocket commented 2 years ago

Done!

zowe / zac

Establishment of Zowe Support Provider Conformance Program #182