Documenting "Freeze period" before/during KCCN NA and improving for future KCCNs

[dims]

We tried it this time, please see https://lists.cncf.io/g/cncf-toc/message/7356

Let's gather thoughts/feedback on what worked and what didn't to write down the policy so it is clear to everyone.

text used in the email was:

Community,
  KubeCon + CloudNativeCon North America is just NINE weeks away! It seems super far right now, but it will creep up on us.  As such, the TOC will be freezing applications for incubation and graduation beginning AUG 26th and ending OCT 31st. We want to ensure that TOC members and the community can focus on preparations for the event and not rush this process as there is often scarce availability by members of the community essential to successful commencement around that time.  Moving forward, we would like to leverage a similar timeframe for all KCCNs moving forward, invoking a graduation freeze 8 weeks before the conference, lifting the week after.

Speaking of graduation, we want to take this opportunity to emphasize the importance of positive, transparent, and kind community relations which is absolutely essential for maturing projects and activities.  In recent weeks we’ve noticed increased tension within the community around sustaining and growing maintainers and leadership within our ecosystem.  A thriving project and group must be capable of demonstrating stability and sustainment in maintainership and leadership as well as a consistent application of stable governance processes no less than 1 year prior to applying. We’ve opened the following issue ([#907](https://github.com/cncf/toc/issues/907)) on the repo to capture your thoughts, views, and experiences prior to applying this change to the graduation process.

Issue: https://github.com/cncf/toc/issues/907

Thank you all for your considerations and passion in continuing to make our community one of amazing people and amazing projects.

And during the period we ended up taking the stance that:

Everything that was not in public comment is being held until date XYZ

Feedback welcome!

[TheFoxAtWork]

Based on community feedback around this process, we're proposing the below modifications moving forward, we would love community feedback on this, especially where it improves clarity and reduces confusion.

• New and to-be sponsored applications to move levels are frozen 8 weeks before each KCCN, resuming the following week.
• Applications with an existing TOC sponsor can continue to progress their Due Diligence (DD) documents and refreshes but will NOT be expedited to meet KCCN scheduling, this includes any ongoing DD creation, refresh, or initial data gathering.
• If a DD or refresh with a TOC sponsor is completed 6 weeks prior to KCCN, they can move to public comment and a subsequent vote if the sponsor deems them ready at that time, this allows 2 weeks for comment, and then vote without causing undue burden
• Any DD or refresh completed more than 2 weeks after the freeze (weeks 6-0) will be held from comment until after KCCN even if their TOC sponsor deems it ready.
• Projects at public comment at the start of the freeze can continue to a vote.

Sample Timeline:

• 8 weeks : new and to-be sponsored are frozen
• 8-6 weeks : existing sponsored applications with completed and refreshed DD can be called for comment then vote
• 6-0 weeks : public comment and votes in flight continue
• 0 : KCCN week
• +1 week : applications, sponsorship, etc. return to normal

[tomkerkhove]

Out of curiosity - What's the reasoning about the time periods? 8 weeks seems a bit long given there are 3 KCCN's every year which means a freeze of 24w/5-6mo per year for new applications?

[makkes]

Given there's two KubeCons each year this means there will be a yearly levelling freeze for projects of 4 months, i.e. for 1/3 of the year there will be no progress in that regard for any project. My concern is that this will just increase the pressure especially on the TOC as anything around levelling up (e.g. DD docs, conversations) will pile up for that amount of time.

Is KubeCon more important than herding projects? This is not a sarcastic question. How can the TOC be freed up from conference work in favor of projects?

[dims]

@tomkerkhove the Kubecon EU does not affect us as much. So this is more for Kubecon NA.

Sorry if it wasn't clear, this is not just for "TOC be freed up". there's work all around by everyone involved in DDs for example.

[xmulligan]

"frozen" is unclear to me. In #917 CRI-O applied for graduation, is that allowed and just no work will be done until after KubeCon or should applications also not be made in the frozen timeframe?

[dims]

@xmulligan folks can open PR(s), it's just that ... "don't expect progress in this time frame"

[tomkerkhove]

@dims That's interesting - What is the difference in terms of workload if it's KubeCon NA or KubeCon EU? I don't see much of a difference, but it might be my lack of knowledge.

[makkes]

Sorry if it wasn't clear, this is not just for "TOC be freed up". there's work all around by everyone involved in DDs for example.

Thanks for the explanation @dims. Maybe the freeze period should be a little more relaxed from "strictly no progress" to "progress based on individual availability". If a certain sponsor would not be affected by KCCN prep they can still move a project's application forward.

[dims]

@makkes Feel free to propose verbiage changes to what @TheFoxAtWork has posted, how would we tweak that to support what you are looking for?

Also note that there are steps where the entire TOC has to do some things (like review the DD's, vote! etc), so there will still be things that we cannot do.

[dims]

@tomkerkhove this is anecdotal, the pressure to do things/deadlines is less for EU (vs US). If this spikes up, we'll have to iterate and figure out if we want to use the same freeze guidelines or something lesser.

[makkes]

Comparing

Also note that there are steps where the entire TOC has to do some things (like review the DD's, vote! etc), so there will still be things that we cannot do.

to

Sorry if it wasn't clear, this is not just for "TOC be freed up". there's work all around by everyone involved in DDs for example.

we might still need to think about how to give TOC folks more time to work on moving projects forward, no?

[makkes]

@makkes Feel free to propose verbiage changes to what @TheFoxAtWork has posted, how would we tweak that to support what you are looking for?

I wouldn't be able to do that until I completely understand the requirements that went into the proposal. I perceive it like this:

1. TOC needs 8 weeks (or 6?) prior to KCCN to fully focus on KCCN preparation.
2. Projects are increasingly asking for expedition of their levelling process prior to KCCN.

As much as I can understand that it's not feasible to expedite the process just for KCCN I would like to understand what the reasons for (1) are and whether it would rather be desirable for the TOC to be able to spend more time on projects than on preparing KCCN. I suppose this discussion goes to the core of what the TOC is responsible for. In my understanding of the TOC's principles favoring KCCN work over actually moving projects forward doesn't quite align with said principles.

[hiddeco]

I would argue that with projects having their own focus weeks before KCCN as well, this policy will result in more pressure as they now have to deal with this and the "unavailability" of the TOC.

In addition to this, the push to get things done for next KubeCon is right after current KubeCon. Which means that you end up in a situation where the loop between conferences will now be closed with all sorts of due diligence that needs to happen in this specific period. Be it either for the TOC, or in preparation of the next conference.

All together, this makes me skeptical about the policy being favorable to anyone, other than members of the TOC themselves.

---

This does not take into account

the pressure to do things/deadlines is less for EU (vs US)

but we might wonder why US > EU. Sounds like making them both as important could theoretically split this pressure :-).

[tomkerkhove]

This does not take into account

the pressure to do things/deadlines is less for EU (vs US)

but we might wonder why US > EU. Sounds like making them both as important could theoretically split this pressure :-).

This is also what I wonder

[hiddeco]

Additional food for thought: making both as important would also help with creating more sustainable conferences. As you will not "miss out" if you do not take the plane from EU to US for a year (and while I am in favor of virtual attendance, this is not the same).

[TheFoxAtWork]

Thank you to everyone for the questions to understand the constraints or circumstances that drove this proposal in order to improve upon it.

For additional clarification to assist in improving the proposed change:

• 8 weeks for new applications without sponsors : undergoing due diligence is non-insignificant amount of work. conducting adopter interviews takes time and scheduling which becomes increasingly difficult the closer to each KCCN we get. This an additional commitment by all parties to perform the work in a more compressed timeline as many projects wish to graduate or move levels right before KCCN so they can announce it.
  • By removing KCCN as a goal post for brand new requests to move levels, we hope to not burn out adopters, TOC, maintainers, and other community members.
• Individual availability : while an assigned TOC sponsor may have cycles, and the community available to support a Due Diligence or refresh with adopter interviews, those interviewees may not be available in the timeline, and TOC members do review the contents of the Due Diligence prior to and during public comment. It is part of why they are due diligence, to understand the project, its status, its progress, direction, and growth, and this takes time. So while one individual may be available, not all parties involved may have the cycles for comprehensive review.
• Up to 6 weeks prior for projects with a sponsors and in flight: Often, DD or the refresh progresses slowly, gathering the data, making it clear, addressing questions, etc. until the data is pulled together. Once together, things begin to move much more quickly. with the two weeks for public comment as an existing set aside after the DD or refresh is done, voting can take anywhere from one day to (not ideal or common) a few weeks to complete. Which provides a two week buffer before KCCN that a project could prepare anything necessary for the event to announce movement.
• KCCN conferences : KCCN EU has begun to grow significantly. While this change is initially focused on NA as an initial effort, we anticipate it to be applied to EU as well, especially given the increased diversity and distribution of our community outside of the US. Each conference (EU, NA, China) should be considered based on the current trend.

For projects that wish to graduate or move levels right before KCCN, we ask they begin the the application no less than 8 weeks before the KCCN however we cannot guarantee graduation or movement by that time.

Our goal with introducing these changes is to ensure DD, refreshes, and interviews all meet the same rigor in their creation and their review as they do when not subjected to the impending deadlines, timelines, and content submission of such an impactful conference.

[tomkerkhove]

Thanks for the clarification!

• KCCN conferences : KCCN EU has begun to grow significantly. While this change is initially focused on NA as an initial effort, we anticipate it to be applied to EU as well, especially given the increased diversity and distribution of our community outside of the US.

For this ☝️, it might be good to rename the issue to avoid this confusion.

[TheFoxAtWork]

adjusted. thanks !

[kevin-wangzefeng]

• KCCN conferences : KCCN EU has begun to grow significantly. While this change is initially focused on NA as an initial effort, we anticipate it to be applied to EU as well, especially given the increased diversity and distribution of our community outside of the US.

I would like to add that we actually has 3 KCCNs now each year, KCCN China is the one besides KCCN NA and KCCN EU.

[kevin-wangzefeng]

It might be clearer if we can phrase as a guideline/recommandation like this:

• For projects wish to graduate or move levels right before KCCN, please start the application 8 weeks before the KCCN.

If I understand correctly, we don't really want to stop/slow down projects from applying for graduation/incubation, but to clarify people's expectation that, regarding the current situation, the closer to KCCN, the more uncertainty of finishing the process before conference.

As for KCCNs, I think most the preparation work shall be done by the KubeCon Program Commitee members? Though there are a lot of people have multiple roles, but theoretically neither the TOC nor TAG is the first person responsible for KCCNs.

Nit: if we can document more guilelines to help the projects get better prepared before applying for graduation/incubation, it would be very helpful and timesaving (less reworking and iteration during the process).

[TheFoxAtWork]

@kevin-wangzefeng thank you for the comments, i've corrected the conference section to reflect consideration for each of the three.

I've also captured your suggestion on the guideline-it reads much clearer and purpose driven.

regarding KCCN prep - You caught it - a lot of people have multiple roles, not just the TOC. Even if we exclude the enormous task of reviewing CFPs as a program committee member, or as a track chair or even co-chair, Maintainers and TAGs are prepping talks for the maintainer track, recording and submitting, many adopter organizations who are eligible for interviews are busy preparing all manner of material, in addition to their own talks and events and their regular work on top of it all. So while no one is the first person responsible, nearly everyone involved in a graduation or movement between levels is significantly busier the 8 weeks leading up to each conference.

I definitely think documenting or providing and estimated timeline of events for projects would be fantastic and help set expectations. Is this something a member of the community can take a first pass at in a PR, especially someone having gone through moving levels and graduating?

[xmulligan]

Looking at the most recent graduations, I would guess an average of 5 months would be good

containerd 16 Oct 2018 - 3 Mar 2019 (4.5 months) TUF 17 Oct 2018 - 18 Dec 2019 (14 months) Vitess 7 Oct 2019 - 5 Feb 2020 (4 months) TiKV 1 Apr 2020 - 8 Sep 2020 (5 months) OPA 27 Aug 2020 - 2 Mar 2021 (6 months) Etcd 23 Sep 2020 - 2 Mar 2021 (5 months) Linkerd 17 Mar 2021 - 10 Aug 2021 (5 months)

[TheFoxAtWork]

So about 5 months for graduations, we can work backwards and estimate roughly 4 months to get the DD done/ refreshed and interviews conducted. Interviews occur sporadically but i believe they are usually done after the DD is considered near-complete. If you estimate a month for scheduling, holding, writing up interview summaries (this assumes the interviewee can be attributed as anonymous interviewees usually have long approval times) that leaves a 3 months for DD.

Estimated timeline:

• Project believes they are ready to move levels
• Project applies to move levels
• TOC sponsor(s) is identified and assigned ~ XX days/weeks since application received
• Coordination with project on DD creation/refresh ~ 1-2 weeks
• DD creation/refresh ~ 3 months
• Interviews occur ~1 month
• Internal TOC review before public comment ~1-2 weeks (not included in timeline, included for transparency)
• Public Comment ~ 2 weeks
• Vote ~ 2 weeks
• Email sent confirming results

@amye & @dims can you estimate the time from application to sponsorship?

Community - when discussions within a project begin about moving levels, what internal checks or discussions do you engage in and for how long prior to applying?

[amye]

"@amye & @dims can you estimate the time from application to sponsorship?" < - Varies widely, as it's up to individual availability on the TOC.

[dims]

reminds me of the SEC Rule 156 - "Past Performance Is Not Indicative Of Future Results"

[xmulligan]

Taking a first stab at writing this up. Feel free to comment/edit. I think it could be added here under a new "Timelines" section. I'm guessing a bit a the spirit of what everyone means so if I missed the mark, please let me know. Also going to add wording for sandbox since that also frequently comes up.

"The TOC makes no guarantees on when/if a project will join the CNCF or move levels.

For sandbox proposals, applications are reviewed from oldest application to newest every two months and the TOC may not have time to get through every application each meeting. The up to date list can be found here and will be carried over from meeting to meeting if not every one is reviewed.

For moving levels to incubation or graduation, projects should plan on at least 5 months or more between the initial application and approval. Due to the increased community demands around KubeCon + CloudNativeCon, new public comment periods and public votes will not launch within 6 weeks of a KubeCon + CloudNativeCon. Project should take this into account when planning completion of their due diligence."

[TheFoxAtWork]

@xmulligan i think a timelines section is appropriate as well, were you thinking of that to go on the proposals page with an FAQ that provides additional information/ background?

[xmulligan]

@TheFoxAtWork yes exactly, are you ok if I open a PR there?

[TheFoxAtWork]

Please do!

[xmulligan]

Opened in #933 let me know what you think about the FAQ I added

[TheFoxAtWork]

closing issue as PR #933 makes this complete