Exploratory Groups - New short lived entities within CNCF

[dims]

Exploration Groups (Tentative Name!)

We should have some time bound working groups with clear entry and exit criteria that explore an emerging area with the explicit intent to provide recommendations to the TOC on how to proceed or integrate into CNCF (if appropriate). These may be cross-TAG, or within a TAG. May produce limited education/awareness content to remove ambiguity in the space. Does not have any organizational authority, only influence. This is more to get folks interested in a $TOPIC in a room, give them a structure to explore and come up with some useful things for the overall community. Exploration Groups could provide a formal avenue for disparate groups to collaborate around a common problem, craft a balanced position and provide recommendations to the TOC, and disband. Because they represent the interests of multiple groups, they are a vehicle for consensus building.

Some initial notes captured in previous meetings:

If any assets are developed as part of the collaboration within an Exploration Group, those assets will be housed in 
an appropriate location as described by the stakeholder TAG(s).

The Exploration Group formation begins with the organizers asking themselves some important questions that should 
eventually be documented in the proposal for the said group.

- What is the exact problem this group is trying to solve?
- What is the artifact that this group will deliver, and to whom?
- How does the group know when the problem solving process is completed, and it is time for the Exploration Group to dissolve?
- Who are all of the stakeholder TAGs involved in this problem this group is trying to solve?
- What are the meeting mechanics (frequency, duration, roles)?
- Does the goal of the Exploration Group represent the needs of the cloud native community as a whole, or is it focused 
  on the interests of a narrow set of contributors or companies?
- Who will chair the group, and ensure it continues to meet these requirements?
- Is diversity well-represented in the Exploration Group?

Once merged, the Exploration Group is officially chartered until it either completes its stated goal, has not been renewed by 
the TOC after a 6 month period or disbands voluntarily (e.g. due to new facts, member attrition, change in direction, etc). 
Exploration Groups should strive to make regular reports to stakeholder TAGs in order to ensure the mission is still 
aligned with the current state.

We've opened this issue to gather more thoughts/ideas from you all.

Many thanks to @TheFoxAtWork for the extensive notes :)

[endocrimes]

It feels like these somewhat ~= "shorter lived WGs"?

I'm curious to see what pain point they would solve, and whether we could make it such that a WG can be lightweight enough to offer the same benefits?

[dims]

could be that we can't get rid of WGs? :)

[riaankleinhans]

According to the current "CNCF Technical Advisory Groups ("TAGs")" document under "Responsibilities & Empowerment of TAGs" it state that:

"TAGs may choose to spawn focused and time-limited working groups to achieve some of their responsibilities (for example, to produce a specific educational white paper, or portfolio gap analysis report). Working groups should have a clearly documented charter, timeline (typically a few quarters at most), and set of deliverables. Once the timeline has elapsed, or the deliverables delivered, the working group dissolves, or is explicitly re-chartered."

However from my observation the only TAG that have used Working groups as “time-limited working groups to achieve some of their responsibilities” is TAG Observability ie. TAG OBS Query Standardization WG Charter

Then we have also legacy "CNCF Working Groups" like the "Serverless WG" which is still active as a WG, not under a TAG. Ideally it should become a TAG in it's own right.

In MOST cases the Working Groups we have today as a subsection of a TAG with distinct responsibilities that are not time bound and is attending to a part of the working group's scope. Ie. TAG CS has several Working groups for Governance, Mentoring, DHoH, which is all permanent structure with a README and Charter.

Out of the TAG / TOC meeting at Kubecon Paris there was proposals around create another entity type for example "Mentoring WG" which a permanent and important structure.

This is clearly not an answer to "short lived entities" question but just highlighting how complex out organizational landscape is and how often we are contradicting in practice and documentation. When we choose to make new entities we should consider the whole org landscape and update our documentation accordingly.

[monadic]

thanks @Riaankl yes the original concept was that TAGs are long lived, and WGs have an exit point defined

[anvega]

On ways to improve how we all work together between TOC and TAGs, I see the ease and ability to take on new work is at the core of that. I've also noticed we're starting to talk about "sociotechnical systems" a lot more, and I think it's a good time to share what I think that really means in our context.

Back when SIGs were smaller groups who managed their own stuff we managed to stay in touch with the TOC and other SIGs on exploratory efforts easily. We were working on the same level of concerns on a smaller number and surface of projects. But as things grew and SIGs turned into TAGs and have grown in membership size, things have got more complicated. Now, if a TAG hits a snag, it takes time to get the message up through the chairs, then to the liaisons, and finally to the TOC before we can move forward again. Plus, it feels like we're only ever talking one way, from TAG to TOC, especially if we need to "green light" something. We're only really talking through our chairs to the liaisons, and the idle time is making many feel disconnected and stuck.

With all the focus on monthly updates for TOC meetings, it feels like there's a big disconnect between that and actual ongoing discussions and the real work we're doing, like coding, research, and working with project teams. It's like the success of the broader members and their initiatives is in the hands of unseen processes and figures whom regular members, and much less newcomers, will ever see or talk to outside of leadership/managerial channels, which just makes everyone feel more isolated and frustrated with their ability to explore and take on new work. It feels to like they are being set up to fail if exploratory ideas or new interesting work require sign-off.

I see many newcomers come up with ideas but they don’t get going on it out on their own because of the formality or structure, but more due to a lack of percept coalition around explorations and experiments.

So, I've got two main ideas to help us work better together and feel more connected specifically on this sort of exploratory effort over any span of time:

1. Let people start their workstreams as long as they stick to their respective TAG contribution guidelines. This would give us all a bit more autonomy and say in what we're doing, along with the ability to do our best work. We get to spend our energy on the work instead of getting bogged down in proposals and procedures. Guidance on how to be organized does help so I'm not advocating for telling folks to freestyle it but at a point where the exploration has a breakthrough or produces an output, it can be assessed for its completeness and value retrospectively.
2. Have the liaisons hang out with the TAGs more and be available to the TAG members, maybe join our TAG calls every two weeks, or have a catch-up or AMA of sorts outside of just talking to the chairs. This helps in two ways. First, it allows liaisons to directly hear about ongoing explorations and fresh ideas from the members Secondly, it benefits everyone who isn't involved in TOC calls by providing them access to the liaisons' broader insights on CNCF and the wider ecosystem.

We need to make sure we're all asking and answering on new workstreams: "What should be happening?" and "What's actually happening?" while ensuring there is some flow efficiency around the actual work and not its description. This way, we can all feel more part of the team and get stuff done together.

[anvega]

Another thing to think about is how having approved these efforts as “approved entities” is that it gives them a kind of spotlight, which can help pull in more people to help out. But then, there's something special about seeing a group just come together on their own and manage to get others excited about what they're doing. They can make use of all the different ways we have to talk about stuff in the organization like this issue tracker and slack to get people interested. Certainly more interesting than processing the intake of proposals.

And if liaisons are hanging out more with the TAGs, they'll be right there to see these efforts up close. They can help spread the word and get more eyes on what these groups are doing.

[TheFoxAtWork]

Some background as i realize it was not provided in the notes:

This proposal spun out of many discussions, most recently at KubeCon Europe, and is one of few proposals to address concerns raised (one proposal was dropped altogether due to lack of interest). The proposal of Exploration Groups was to create a structure for those interested individuals to come together around a focus, with an established group, and a concrete set of outcomes that is time-bound. They may spin off into working groups or eventually form a TAG on their own, but it initially ensures consistency for new and existing contributors across TAGs in setting expectations for how the CNCF and these overarching groups operate as well as take on new work.

Think of it as a growth structure. an idea may start with forming an exploration group. the exploration group concludes and says - okay we really need more work to happen here lets create a working group under the TAG, or this really needs to be its own TAG, or we've looked into this and there isnt significant work here therefore the whitepaper is sufficient to address this.

The TOC and TAGs have been meeting for the past several KubeCons as well as on our monthly calls to address several concerns associated with stagnation, broad scope, contribution challenges, direction, etc. of the TAGs. Not all TAGs experience all of these or some of them. It varies greatly from TAG to TAG. This makes the contributor experience in TAGs difficult to navigate, understand what a TAG is doing, and participate meaningfully.

In the course of these meetings, the TOC requested a review of the TAG Charters and work to understand where we have variance, where we have spikes of work, where we have efforts being diluted as a result of limited contributors, and more. It was noted that one area of work that keeps coming up are new technologies and their impact, inclusion, and integration in cloud native.

How this shows up is usually a request to create a new TAG being made on the TOC repo with very little understanding of the area being proposed in the context of cloud native. AI is an example of this, as is Web Assembly. The TOC has been moving these requests into an aligned TAG, but often as a working group. We're effectively redirecting interested parties to groups that may or may not have the capacity to manage and take on the additional orchestration and facilitation of these.

Right now this is just a proposal driven by feedback and input from the TAGs on the volume of work they have. If this isnt a structure the TAGs feel they need called out in their charters, that's fine. We've tried tackling this a few different ways in the past and nothing really stuck. Some working groups live on forever as they cover specific sub-domains of focus under a TAG, and some eventually wind down.

[angellk]

@dims within TAG App Delivery, we are looking at creating a WG for Infrastructure Lifecycle - it would be good to discuss whether that is better started as an Exploratory Group vs a Working Group

[raravena80]

Some thoughts that came up in the TOC meeting today (04-02-2024): It would help to document the path from exploratory groups (short lived) to working groups (long lived), and later TAGs(?) This in addition to documenting what exploratory and working groups are.

[rochaporto]

Another comment from today's TOC meeting, we already have WGs that are not tied to a single TAG: wg-serveless, cartografos.

Adding to that, the questions being asked above are similar to the ones in the process for WG creation under kubernetes: https://github.com/kubernetes/community/blob/master/committee-steering/governance/wg-governance.md#creation-process-description

which highlight the answers to the need for a clear entry and exit criteria. Do we have this clear for the existing WGs, especially the ones existing for several years? Should we review those before starting a new type of WG?

[cathyhongzhang]

I am wondering if there is a need to create and define this new "exploration groups" entity. Existing working groups can be time-limited or stay for a long time. Defining a new entity may make it more confusing to people. I think what we need is something like "horizontal TAG" which involves technology areas across multiple vertical TAGs. Serverless WG is such an example. AI WG is another example. Both WG's working areas involve runtime, network, storage, observability, etc. Without a "horizontal TAG", we resulted in having an independent WG not hosted under any TAG, such as serverless, or having a WG that is hosted under multiple TAGs which could have operational complexity. What we need is probably to define how a WG evolves into a "Horizontal TAG".

[zanetworker]

I see two distinct problems:

1. Identify the means "for disparate groups to collaborate around a common problem" (cc @dims). A concrete example here is #hosted-controlplanes. Those groups typically have a problem, have solved it in different ways (in their own bubbles), now they would like to explore how to streamline solutions, approaches, and discussions so that others looking to solve the same problems in the future don't have to start from scratch, or simply accelerate collaboration and create net new opportunities.

2. Assuming the folks from 1. formed a WG (that has been the default answer so far), i.e., had someone/something guide their souls to a place where they can meet and discus ideas, in a formal context, the next problem is define where the group should go next. In somecases, where the groups touches on many areas in the ecosystem (e.g., AI, Serverless, ...), the question becomes how to define how a WG evolves into a "Horizontal TAG" (cc @cathyhongzhang). In others, the group maintains focus on a single topic, it's clear but needs a resting place, where should that be?

[TheFoxAtWork]

Slack channel for discussion #tmp-eg-discuss