Zowe squad proposal to accomodate new contributions & promote further integration

[solsu01]

New subprojects:

1. Zowe Client SDK
2. Zowe Explorer

ZLC approved presenting this at SHARE as an "idea", and for the ZLC to meet at SHARE to whiteboard how to organize the squads to facilitate the addition of these new sub-projects.

Sharing details in the ppt here. zowe_new_subprojects.pptx

[1000TurquoisePogs]

I think this is a good first attempt at categorization since it closely aligns with what we have been telling people for the last year, let me explain...

One of my favorite "what is zowe" summaries is that one slide we use which breaks down into 4 pillars of 2 categories

(I know, this one is a little outdated... was trying to find a 'simple' one)

Client: ui -> your "hosted tooling tribe" (mobile can be as distributed as workstation, need a different word than hosted) cli -> your "workstation tooling" ?

Server: apiml -> "shared services" microservices/servers -> also "shared services"

I like the 4 pillar model is that it's been a proven simple way to explain Zowe. I think it got us through 2019 and over the initial confusion people had in 2018.

Here, the big difference seems to be that this is a 3 pillar model, but I don't think that changes much.

The additions of mobile and python variant of cli sdk to me also does not change that model... maybe one could argue differences between mobile ui and browser ui but on many levels they are similar. I could see the image above substituting the "web desktop" title for "ui" if we have a consistent story.

The concept of tribes is interesting but I think it makes sense that they fit the 4 pillar model. For example: what would be the impact of client sdks being in a different tribe from cli? Are they really independent? Does the independence help us solve problems better than before?

Ultimately the discussion I want to have is how the tribe model would change day-to-day activities.

[solsu01]

Thanks @1000TurquoisePogs. I've taken your feedback and also solicited feedback from many others in Zowe to get a reality check. Will share a new rev soon.

[solsu01]

After taking into account @1000TurquoisePogs's feedback and soliciting feedback from others in Zowe, I've boiled down the proposal to match near-term realistic goals with reduced disruption to the way squads operate today. To reiterate, the intent of this proposed structure is two-fold 1) align the incoming contributions with the tech they're currently coupled with and 2) put us on a path towards achieving greater synergy and re-use between squads.

zowe squads_ss_v3.pptx

[1000TurquoisePogs]

I like this version more, thank you. There are still some details I don't know about, such as whether JP would lead Desktop/ZSS work. But, I think you got the Architect boxes right and I'd be glad to work with Dan & Petr & Joe on figuring out Zowe.

The "Whats not working" slide has me wondering about how to fix a few, and why others are a problem:

Conformance is lacking specificity

Is this true? I thought our conformance had specificity, but maybe I am the wrong person to ask. It has a lot of "thou shalt" and "thou shalt not" which I thought was specific. There can always be more but we don't want to limit creativity.

Lack of new innovations beyond the initial tech

This is sort of true, but I think a lot of the squad efforts have currently been on implementing asks from vendors/users/community that aren't innovative but are required, such as SSO, MFA. But, yes, we need to keep the innovation up 👍 But, sometimes innovation comes from not following a schedule, such as hackathons. We should consider if the structure allows for freedom to explore & create new things.

Overlap/competition with other IBM tooling

What does this mean and why is it a concern? Are there specific open source IBM tools that we are ignoring? Since we are an open source community, we should not be worried about competing with closed source. Linux did not succeed by being concerned about Microsoft. There's the copyleft versus permissive concern on whether open source can/should donate back to closed source... personally I'd like what we provide to become commonplace, but we must obey EPL-2.0 too.

[solsu01]

Thanks for the feedback @1000TurquoisePogs !

Overlap/competition with other IBM tooling

This is something Matt H mentioned on the ZLC call from a few weeks ago. It may not be something that Zowe can solve and rather something IBM internal.

[jmertic]

This proposal really lacks a connection between the problem statements and the solution. Maybe it's implicit, but I'd suggest it making it explicit.

That all said, if I'm taking each problem statement explicit and trying to grok the model as the solution...

• Contributing-company based silo'ing

I'd agree this is a current state, but I'm unclear how what appears to be a multi-tiered hierarchy helps reduce silo'ing.

• Lack of new innovations beyond the initial tech

Are you talking within the project or downstream? Is that an implicit goal to have "new innovation" within in Zowe. And again from above - how does putting more hierarchy help this?

• Lack of integrations and synergy between components.

I'd also agree this is a current state. Still stuck on more hierarchy helping this.

As a solution, why couldn't the ZLC facilitate this? In most other projects there's a concept of a TAC or TOC that serves the purpose of creating this forum and letting the connection points naturally emerge.

• Lack of overarching architectural runway

Outside of +1ing the hierarchy bender I'm on, I think a more basic question that needs answered is "What is Zowe and what isn't Zowe" ( definition of scope ).

• Conformance is lacking specificity

I'd echo a bit of what @1000TurquoisePogs said, but also just scratching my head on how a squad re-org fixes that ( irregardless of hierarchical or flat structures ).

The big challenge with conformance is that everyone wants to keep the bar really low to spur more people completing it. Specificity is an anti-goal. Squad reorg doesn't change that view.

• Lost touch with users

Reorgs don't help that problem. Consistent and clear messaging do.

• Overlap/competition with other IBM tooling

I'd +1 @1000TurquoisePogs - if IBM views Zowe as competitive there are bigger issues, but they are out of scope of the project.

Sorry to poke holes over all of this - it likely could be there is something implicit that's not clear, but I'll say that doing a re-org for the sake of "shaking things up" like what it appears here tends to not help solve problems which are more deeply rooted...

[1000TurquoisePogs]

I think @jmertic s points do need to be addressed before we take action on this, because organization structure impacts everyone. What I don't want is to reorg only to later realize it didn't gain us much more than process. I've been open to the ideas but skeptic about if all of the goals will be met. I think some of them will, some of them wont, so we shouldn't judge our (ZLC's) success on this.

For one, Innovation is a good thing, but it should be put into the perspective of the OMP as a whole: If we have a new cool idea, is it Zowe or is it OMP? There is no reason for other OMP projects to be strangers. Apache & Tomcat are both projects that are well defined because they just kept iterating, improving, perfecting their original purpose. I know this is a bit tough for Zowe since it's a multi-faceted approach from day 1, and so the idea of "integration" does help us to in that regard.

The big question to me is will this solve siloing? So far, we've done a good job of increasing silo-ing in that we've been able to better define what components of zowe you should use under what circumstances. This is not a bad thing in that regard. Should components make use of each other to reduce re-implementation? Absolutely, and this chart looks to help that, but it will just connect the dots rather than eliminating silos. For example again, SSO: it is a cross-group effort. This is good. But at the end of the day if you want to do X you still want to talk to group Y. We should identify more of these items, but I do not think it would change the fundamental components at play. The one thing we can do is not make this siloing worse, by making sure that innovations & inventions are scrutinized on whether they fit into Zowe or should become a sister project at the OMP level.

[solsu01]

Thanks for the detailed feedback @jmertic and @1000TurquoisePogs. Good points all around.

Keep in mind that this proposal is not really a "re-org" at this point. Squads stay as-is with the exception of Zowe Explorer being split into a separate squad. Outside of that it's just a recognition of the need to assemble a group of SMEs from each component to think about re-use, and synergy.

We agreed to go forward with v3 of this proposal at the last ZLC call. So I've been communicating this to the squad leads and soliciting feedback. In the interest of not going back & forth, perhaps we could bring up this topic again on the next ZLC call while we continue to execute on it.

[solsu01]

Attached is another rev based on the feedback here. I made it clearer on what the squad evolution addresses and what is out of scope. zowe.squads_ss_v4.pptx

[Joe-Winchester]

@solsu01 - Tks for this deck. I am very keen that this is seen as a way to increase the value of what we deliver by tackling issues that span different technologies in a common way, through shared sprints, shared backlogs, and also shared communication to sponsored users and folks working on the issues. Current items that customers are asking to be tackled in Zowe that don't fit into existing squad structure 1:1 cover (not a complete list)

• Security. covering SSO, MFA, use of SAF keyrings, ability to operate these across all customer landscapes of JWT generator and MFA offering, ... This covers a number of areas (desktop, APIML, CLI, Zowe Explorer).
  • Containerization. covering docker to allow USS portions to be run off host, on host under ZCX or zLinux, run on zD&T to increase performance of APIML and Java Explorer servers, leading to investigation of HA through K8S and Helm, ...
  • Support. covering ensuring untampered integrity of Zowe distribution, improved diagnostic scripts, more formalized FAQ forums, blog/videos/docs from squad members to allow increased ability of non squad members to provide support, possible conformance criteria for SI wanting to provide support, ...
  • z/OS GUI experience - covering MVS/USS/JES Explorer - File Editor - Zowe Explorer. There is synergy that can occur here, possibly looking at Zowe Explorer inside Theia being in base Zowe, and what this would mean for the MVS/ISS Explorer and File Editor.

There are others I've omitted, but in the new structure do you view the "Integration Architecture Squad" as being the glue between all of these initiatives that are cross squad ? If each squad does separate PI planning then what do you envisage is the best way that this kind of cross squad work gets PI planned and committed to be worked on (assuming it's the right stuff to work on) ?

[solsu01]

Since there was agreement on a subset of this proposal to formally split the Zowe Explorer related work into a separate squad, I've put together a path of execution to make this happen. The other parts of the proposal still need to be discussed and require consensus before we can proceed with execution.

New Zowe Squad: IDE Mission:

• To take ownership of all IDE tooling topics for all of Zowe
• To maintain and grow the Zowe Explorer extension for VS Code.
• To deliver Zowe Explorer as part of Zowe Core including an offline package delivered on Zowe.org similar to the offline package for Zowe CLI
• To formalize extensions to Zowe Explorer by means of well documented extension points and if applicable, conformance rules must be defined.
• To work with other squads in Zowe to ensure re-use, and synergy across Zowe components

Eligibility:

• The initial committers must be existing committers from the CLI squad. Afterwards, committers to the IDE squad can be added via IDE squad consensus and meritocracy.
• Squad members are allowed to be a part of more than one Zowe squad as long as they can reasonably contribute to both

Squad leadership:

• Once the squad members are confirmed, they elect a squad leader who will perform duties similar to other Zowe squad leads such as setting priorities, and leading backlog refinement/planning. If there are no volunteers for the squad leader position, ZLC will appoint an interim squad lead until the squad can elect their own.
• The squad will also choose a scrum master to ensure smooth operations of the squad and general facilitation.

Execution path - @solsu01, @MikeBauerCA and @MarkAckert will help drive this to execution

[armstro]

Thanks Sujay for writing this all down - we can add to zlc agenda for Apr 14 if we want to discuss there - some comments:

-Regarding "all IDE topics" - didn't we think the App Framework squad+Mobile incubator would include an IDE in their mission? Is there a way qualify Zowe Explorer not multi-tenant? (I made that up.)

• Eligibility - do we want to loosen the "only committers" from CLI to try to cast a wider net for members (or as a contributor and willing to be mentored to become a committer)

[solsu01]

-Regarding "all IDE topics" - didn't we think the App Framework squad+Mobile incubator would include an IDE in their mission? Is there a way qualify Zowe Explorer not multi-tenant? (I made that up.)

Hmm, I was going off a comment @Joe-Winchester made about this squad caring for IDEs in general. I think we said there could be potential reuse of components between the app framework/mobile squad and a potential multi-tenant IDE. Generally, though I'm not sure if this limitation would work as there is already "multi-tenant" guidance for Zowe Explorer - https://github.com/zowe/vscode-extension-for-zowe/blob/master/docs/README-Theia.md

Eligibility - do we want to loosen the "only committers" from CLI to try to cast a wider net for members (or as a contributor and willing to be mentored to become a committer)

My thought here was that we wouldn't want to bother with screening recruitment outside of the squad. If it's initially staffed with folks who are most certainly already committers contributing to Zowe Explorer, then once the squad is established they can screen and add as many new committers as they wish based on meritocracy (I didn't want ZLC to own the evaluation process of whether someone has meritocracy to join). Thoughts?

[armstro]

Re IDE - I am likely confusing the IDE and SDK discussions. I think we said App Framework, CLI and Explorer each had a "SDK" dimension .....I'm fine if we say Zowe Explorer squad takes the lead on IDE and can be shared across Zowe.

Yes we can start with only committers for Explorer - let the squad recruit.

[1000TurquoisePogs]

Sounds like the uncertainty has been solved, but if you mean to put this on the record anywhere you may want to clarify wording:

To be eligible to join the IDE squad, you must already be an existing committer on the CLI squad who is a contributor to the Zowe Explorer

Once the squad is established, new committers may be added by means of squad consensus and meritocracy

Sounds more like:

For the formation of this squad, the initial committers must be existing committers from the CLI squad. Afterwards, committers to the IDE squad can be added via IDE squad consensus and meritocracy.

[solsu01]

That is correct @1000TurquoisePogs.

[solsu01]

IDE squad creation is being tracked in https://github.com/zowe/vscode-extension-for-zowe/issues/732

[armstro]

Issue closed with the creation of CLI SDK and Zowe Explorer squads - will open new issue if/when additional squads need to be formed