search

kubernetes / community

Kubernetes community content
Apache License 2.0
12.04k stars 5.18k forks source link

Refining the way we communicate deprecations/wide-reaching changes to the project #5344

Open justaugustus opened 3 years ago

justaugustus commented 3 years ago

As we go through deprecations and infrastructure changes in the project, it might be a worthwhile exercise to assess and refine the way we communicate them.

I can think of a few recent examples that caused some panic and required additional lift from contributors to reframe or contort/extend support to accommodate:

We should consider what it means to turn down a service, piece of functionality, or kubernetes/kubernetes-adjacent system and type of impact it may have for consumers.

Without policing contributors, as maintainers of the project, we also have a responsibility to users to be careful and deliberate with our communications outside of the project, whether it be Twitter, Hacker News, etc., etc.

So how can we improve?

I think depending on the scope of a change, the following SIGs should be involved in crafting comms:

With @kubernetes/sig-contributor-experience to assist with consistent delivery.

I'm curious to hear everyone's thoughts here.

cc: @kubernetes/steering-committee

parispittman commented 3 years ago

can this be added to the release team communication role with assist from contributor-comms cc/ @mbbroberg for amplification, promotion, and education?

justaugustus commented 3 years ago

xref note to k-dev + Steering: https://groups.google.com/g/kubernetes-dev/c/B0mPMpDLVk4

Katharine commented 3 years ago

My generalised take on the latest adventure, without attempting to police personal contributor tweeting:

The people who write end-user-facing release notes (particularly for changes with action required, e.g. deprecations) should probably not be the the people who wrote the code: they carry a substantial amount of context that the average person does not have, and that is fairly likely to leak in to what they write. At the very least, some people specialising in communications or docs should probably carefully review all such notes.

justaugustus commented 3 years ago

can this be added to the release team communication role with assist from contributor-comms cc/ @mbbroberg for amplification, promotion, and education?

@parispittman -- Yep, I think that sounds great, with that caveat that the Release Comms subteam can be a conduit to merging the various communications streams, but the respective SIG/subproject/component area is on the hook for providing the context that should communicated.

justaugustus commented 3 years ago

At the very least, some people specialising in communications or docs should probably carefully review all such notes.

THIS. THIS. THIS.

jeremyrickard commented 3 years ago

Big +1 echo to what @Katharine said.

can this be added to the release team communication role with assist from contributor-comms

I think so, but like @justaugustus said I think the content responsibility of the owning SIG and it's a partnership with sig-docs. To be honest (having been through a few cycles on the release team now), I think we struggle sometimes to get important things like docs and major themes in/ready sufficiently early to make sure we can communicate things correctly before they show up and hit the wild. We'll add this as a retro item for the 1.20 retro to see what we can discuss and get better for 1.21!

dims commented 3 years ago

I don't think we need steering sign off on this. Cc is fine :) thanks for that!

reylejano commented 3 years ago

I believe that deprecations in a release should be identified by the appropriate release team role as early as possible so that the appropriate sig can prep, review, release the communications. The timeline to release communications should be cognizant of the availability of the changelog prior to the official release date.

celestehorgan commented 3 years ago

As a general rule: industry standard is to communicate a minimum of 6 months out (1+ year is better) from date of deprecation to date of removal. It's considered polite to communicate a schedule as soon as the descision is made, I.e.

Which is what the console log does: the minimum is to communicate timeline and replacement if any.

Standard is also to call out any deprecations in a release notes section (i.e. under major themes, a section titled Deprecations). A good idea is also either writing a blog post or having a section on the website for deprecation notices.

justaugustus commented 3 years ago

I believe that deprecations in a release should be identified by the appropriate release team role as early as possible

@reylejano -- to refine this a little: the owning SIG should signal to the Release Team that a deprecation is happening and that we all need to help prepare comms for it. The onus SHOULD NOT be on SIG Release to sift through issues and PRs to identify something that a SIG reviewer/approver [sh|w]ould have direct context on.

jeremyrickard commented 3 years ago

I believe that deprecations in a release should be identified by the appropriate release team role as early as possible so that the appropriate sig can prep, review, release the communications

I agree, with a caveat. We tracked a couple of deprecations this time around, but the dockershim one wasn't actually in the 1.20 Enhancement Sheet. The current enhancement collection/tracking is really dependent on kubernetes/enhancements issues. The dockershim KEP actually went in as a PR w/o issue and it's easy to miss this stuff. The release team can't sift through all the PRs and stuff that go in to find these, we need to have that affirmative confirmation that something is going to be deprecated (and hence in the release) so we can connect all those dots, vs trying to connect all the dots and make sure we are covered at the end of the release.

lachie83 commented 3 years ago

Just an idea - Could we leverage the Comms team as part of the release team to manage deprecation Comms for that release? They could potentially work on these announcements prior to the release date once code and docs have made it into the release.

kikisdeliveryservice commented 3 years ago

This time around, I specifically tracked deprecations in the tracking sheet (they that had open issues and actively participated in the enhancements process) bc I wanted to be able to highlight them to docs or comms at a glance. I think it's worth mentioning that all KEPs need issues and that non-trivial deprecations probably need a KEP (and a tracking issue/participate in the release process). Also strongly agree that it's simply not possible (or desirable) for the RT to dig thru PRs to match things up - we need authors and sigs to actively & fully participate in the release process.

That said, I think we should consider having an explicit deprecation "graduation" criteria in the KEP template. We need to think about requiring what we need to effectively communicate deprecations in the KEP itself - those are equal requirements to k/k prs imo. That could include: officially announcing the intent to deprecate with help from comms, some guidance of x releases between that initial announcement and beginning the rollout, an explicit rollout calendar with all required comms milestones, etc.. (I made this up, I'm sure people have better ideas 😉 ). I think it's well within our ability to think this through and come up with a way to adequately track what we collectively need, ensure it gets done and have that reflected in the KEP (which is the durable artifact for features/deprecations anyway - why not use it?).

Happy to talk about the above more and brainstorm if anyone is interested!

OmerKahani commented 3 years ago

I want to give my personal view: Currently, the docker depreciation is frightening, as I am not certain what I should do. How can I test if my cluster is using docker? Should I start testing my application using containerd? How should I do it (set up nodes with only containerd)? What would be the upgrade path for an in-cluster upgrade?

I think a guide for end-users who are not reading the KEPs, and maybe even not know what CRI is along with every depreciation, can help make it less frightening and more understandable. a guide that is very operational opinionated.

justaugustus commented 3 years ago

@OmerKahani -- these resources should be helpful:

sbueringer commented 3 years ago

I think we have at least two distinct target audiences for release notes: Kubernetes cluster operator and Kubernetes users.

The vast majority of the changes are only really relevant for the operators. Maybe it would help to make it easier to filter out user-facing changes. The docker deprecation is not an ideal example of this, in most cases it only affects the cluster operators, except when somebody uses docker build with the host Docker socket.

We're operating an internal Kubernetes-as-a-Service platform and although I (as an operator) find the official release notes pretty useful, the subset which is usually relevant to our users is maybe only about 5% of the whole release notes.

makkes commented 3 years ago

@justaugustus I personally strongly believe that what would take a lot of fear from people is a migration guide answering the question of "How do I migrate my existing K8s cluster setup with Docker to containerd or CRI-O?". Then people would see a clear path forward. Just my $0.02.

wilsonehusin commented 3 years ago

I think we have at least two distinct target audiences for release notes: Kubernetes cluster operator and Kubernetes users.

I totally agree with this. I sometimes get confused myself on who this document is speaking to and where the boundaries are. I think it would be useful if we can further disambiguate this in announcements.

sftim commented 3 years ago

Following https://github.com/kubernetes/community/issues/5344#issuecomment-737751369, @makkes I agree and recommend opening an issue in the website repo. (I can or you can).

justaugustus commented 3 years ago

Kubernetes cluster operator and Kubernetes users.

@sbueringer -- it might be helpful to define terms here...

Where do you draw the distinction between "cluster operator" and "user"?

(Maybe naïvely) I would consider "cluster operator" ~= "user". Concretely, if I'm providing a serving some set of applications to a consumer, the underlying infrastructure should be invisible to them (and the Kubernetes release notes become irrelevant from their lens).

justaugustus commented 3 years ago

Tangential to this issue, maybe a valuable exercise is defining personas?

(and maybe that work already exists...) cc: @tashimi @hpandeycodeit -- SIG Usability

justaugustus commented 3 years ago

/sig contributor-experience release /area release-team /milestone v1.21

parispittman commented 3 years ago

@justaugustus the docs team did persona development. They are defined in the glossary now. Cluster operator, platform developer, etc. unless you are looking for something else/new. https://kubernetes.io/docs/reference/glossary

justaugustus commented 3 years ago

@parispittman -- no, that's perfect to potentially frame buckets of release note "types" (if that's something we decide we want to slice against). Thanks!

sbueringer commented 3 years ago

@justaugustus I think it's hard to differentiate sometimes.

My personal experience: We're providing the cluster, our users/customers get a kubeconfig file and can then almost do whatever they want with their cluster (via kubectl, not via ssh). Providing includes: creating/updating/deprovisioning the cluster and monitoring the control plane and node health (this is where it get's blurry :)).

I think this somewhat aligns with what public cloud providers like GKE/EKS/AKS offer.

To me, users mostly care about things like API changes and deprecations (e.g. extensions/v1beta1 => apps/v1). What they usually don't care about are things like kubeadm changes. In my experience they usually just don't have the knowledge to interpret all the other "internal" changes.

Regarding the glossary. In our case:

Maybe tl;dr is: above vs below the Kubernetes API (in the sense of kube-apiserver), but that oversimplifies it quite a bit

I'm not sure if it's a a good idea to separate release notes like this, but just wanted to bring up the idea.

makkes commented 3 years ago

Following #5344 (comment), @makkes I agree and recommend opening an issue in the website repo. (I can or you can).

I created https://github.com/kubernetes/website/issues/25371. This is my first issue on that repo so please bear with me in case it's not as complete as expected.

kikisdeliveryservice commented 3 years ago

I think that we're combining two issues here: 1) how to ensure that deprecations will be adequately communcated from the authors/sigs to the rt to the community and 2) what kind of communication that should be.

In practice, without having number 1(which I talked about earlier) we simply don't have the room to figure out what 2 should look like bc the comms that various deprecations need can only be standardized to a certain extent - some will vary based on impact, but all will require collaboration between the sigs/authors/rt/docs/comms which takes time in a already busy release cycle. If we want more precise or nuanced comms and strategy, we really do need to come up with how this should work to guide sigs and authors and ensure that there's adequate time to get it all done.

vrock28 commented 3 years ago

Usually the EOL for a feature or product gets done by PM’s in collaboration with other teams like comms/doc etc. The best way i have seen it work is using a cascading method where the sig associated with the change (like dockershim deprication) could communicate to other sigs well before in hand. Each sig would then get a chance to evaluate the impact of the change and have a write up/ Communication on any potential gotchas that the customers should be aware of. This way, we would end up calling out all potential impact that the major change would have on customers.

The last thing is to give a heads up to customers on the time they would have before the major change becomes the default behaviour on the platform. My$0.02

OmerKahani commented 3 years ago

@OmerKahani -- these resources should be helpful:

* Blog post: https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/

* FAQ: https://kubernetes.io/blog/2020/12/02/dockershim-faq/

Thanks,

I used the docker as an example, I can make the same list for the depreciation notice in 1.16.

I wanted to emphasize that this kind of notice should come with a tactical guide including:

rdelpret commented 3 years ago

Most of the folks in this thread stay up to date with this stuff and at this point in time this news is at the forefront. However, I think it's important to come up with a mechanism to remind and inform operators who may not hit these deprecations for another year or more. The kubectl warnings have been nice, could there be some mechanism for doing the same with these larger deprecations?

sftim commented 3 years ago

could there be some mechanism for doing the same with these larger deprecations

If there were a ClusterWarning (new kind that I made up), or possibly an Event in the kube-system namespace, kubectl could have the smarts to fetch it and display it to callers when attached to a TTY.

Client side, your kubeconfig could have a mechanism to record an “I've seen this, ignore it until date X” note. The overall effect is to let kubectl and other tooling report when your cluster wants to warn you about something. Once you're sure it's dealt with you can use the API to DELETE the warning.

If that sounds interesting it should become a KEP - lots of technical changes and it will be a few years before that mechanism could be useful. I'd be interested in subscribing to any KEP around this idea, but not driving it.

mkorbi commented 3 years ago

Looking at all the input I would assume we need to have a kind of process around it. I think from the release team the enhancement peeps are a good first place to identify if a change (imo it is maybe not only about deprecation) can call some earthquakes in the community.

This could be brought up through the release meetings and if the release team agrees the comms team takes over. Contacting the SIG, discuss pre-announcements, and maybe some in-depth details about what does this mean for the community.

Sorry, directly end up in problem solving mode :D

salaxander commented 3 years ago

I'm a new contributor (release team shadow) so feel free to take anything I say with any number of grains of salt you choose :)

I agree with what @lachie83 said and I think this is something that would be great to have the comms team involved in, I think it's something we could really help out with

Secondly, I really appreciate that this conversation is happening to begin with. This is a community that cares about it's members far more than many that I've been a part of ❤️

dims commented 3 years ago

fyi, context for the record:

mbbroberg commented 3 years ago

As a heads up, contrib-comms is dedicating this week's meeting to talking through this scenario to plot out how we can be helpful. cc @parispittman.

jeremyrickard commented 3 years ago

We had another sort of pop up yesterday with https://github.com/kubernetes/kubernetes/pull/97057

The TL;DR on this one is that ExecProbe timeouts were never honored/enforced. 1.20 fixes this bug and introduces a feature gate that defaults to true. With the change, if not specified, timeout is 1s (the default that has always been there, but not enforced).

While this is a bug fix, this ends up really being a change to "default" or "expected" behavior since it's been going for so long. The reporter discovered this because they had some probes defined that no longer worked within the defaulted 1 second and it took them a while to determine it from the release notes from the beta2. Changes like this would be good to have more comms around, I think. It's not explicitly a deprecation, but it is change in behavior that could have wide reaching impact on people, especially if they miss the blob in the release notes. We took the initiative to add to the release note for extra clarity and added to the release blog.

dims commented 3 years ago

FYI news from Docker/Mirantis: https://twitter.com/justincormack/status/1334976974083780609?s=21

chris-short commented 3 years ago

I think involving contributor-comms earlier than later is highly advised. I was not able to get too heavily involved due to workload and medical appointments. But, i definitely should’ve gotten someone from the comms team attention too.

I do think forming a process for communicating potential hot potato issues can be improved in literally every project I’ve been a part of.

On Sat, Dec 5, 2020 at 15:15 Davanum Srinivas notifications@github.com wrote:

FYI news from Docker/Mirantis: https://twitter.com/justincormack/status/1334976974083780609?s=21

— You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub https://github.com/kubernetes/community/issues/5344#issuecomment-739396681, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKUD6IO3DAMHBZW2NNUY6LSTKIHFANCNFSM4ULMWNGQ .

--

Chris Short He/Him/His Advisory https://advisedbyshort.com/ | Web https://chrisshort.net/ | Newsletter https://devopsish.com/subscribe/ | Twitch https://www.twitch.tv/redhatopenshift | Twitter https://twitter.com/ChrisShort | LinkedIn https://linkedin.com/in/thechrisshort

mbbroberg commented 3 years ago

Building out communication systems that are helpful in these scenarios is definitely within scope of contributor comms. Looking over this scenario, I think collaboration with release is the right entry point to catching news early and being part of the messaging after the fact. We're piloting our connection points for 1.21 through the conduit that is @chris-short ⚡

sftim commented 3 years ago

:thought_balloon: would it make sense to convert this to a GitHub discussion?

wilsonehusin commented 3 years ago

The 1.20 release team had release retrospective last week and we discussed about gathering and publishing major themes as part of a release. Some discussion points:

fejta-bot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

puerco commented 3 years ago

/remove-lifecycle stale

justaugustus commented 3 years ago

xref: https://github.com/kubernetes/community/issues/5730

BenTheElder commented 3 years ago

I'll be that person: .... was anything refined?

I also see a lot of asks for super detailed messaging and content around this, but not people offering to staff this ... This project still depends quite a bit on the effort of volunteers, even those that are paid to work on the project are volunteering to do work outside their nominal work-role.

Writing good docs / blogs / guides is hard work, I would love to see more paid professional technical writers pointed at SIG Docs :-)

divya-mohan0209 commented 3 years ago

+1 to what @BenTheElder said.

In addition given that these are major changes, it would make a lot of sense for the Release team (in general, & Release Comms, in specific) to have a heads-up so that the delivery can be appropriately coordinated & staged. This will be helpful if there are multiple changes going in a particular release.

Release Comms lead & shadows can then liaise with EO's, sig-docs, sig-docs-blog, and sig-contribex before the feature blog freeze towards getting everyone on the same page & understanding what a good way towards putting out the word would be - staging the content out in different blog posts, covering it all in one etc.

Also while sig-docs, sig-docs-blog, and sig-contribex should be in charge of reviewing, from past experience the folks closest to the change should be the ones authoring the blog/s content. This would ensure that unnecessary panic is avoided by covering all bases including any alternatives etc.

justaugustus commented 3 years ago

From v1.22 retrospective: /assign @jlbutler /milestone v1.23

nikopen commented 3 years ago

how about a dedicated & curated page under https://kubernetes.io/releases/ ?

several levels of importance that I see: Super-wide-reaching changes (e.g. dockershim), other Critical Cluster-Breaking Stuff , etc

Super-wide-reaching changes (e.g. dockershim) can definitely utilize pro-active blog writing that summarizes the Why and How It Impacts My Clusters parts on the very top
great examples 1 and 2, were written very quickly after the announcement but if created before it would ease part of the panic (maybe)

other Critical-Cluster Breaking Stuff goal is to pretty-print and summarize the critical cluster-breaking stuff (I would think). cherry-picking urgent upgrade notes and some wide-reaching deprecations, highlighting the topics can be a good start. https://kubernetes.io/releases/notes/ just redirects to the long CHANGELOGs which are a bit hard to digest TBH....

other Important Deprecations
etc

coordination can be shared between release-team -> communications-team & docs-team & release-notes-team i.e.

just some ideas...

reylejano commented 3 years ago

how about a dedicated & curated page under https://kubernetes.io/releases/ ?

Similar but not the same, there is an API deprecation guide on https://kubernetes.io/docs/reference/using-api/deprecation-guide/ This is one of the only pages that breaks the Kubernetes style guide - avoid statements of the future

In 1.22, the Release Team and SIG Docs members published an API and Feature Removals blog after the start of the 1.22 code freeze. From v1.22 retrospective, this will be part of the Release Team's communication efforts in the release cycle

sftim commented 3 years ago

I'm :+1: on the idea of moving (more like cloning) key release-related information into a page within https://kubernetes.io/releases/ - especially if we can focus on the really important stuff. That page could link to relevant blog articles, so not just:

  • Dockershim is deprecated

and more like:

Notable updates for Kubernetes v2.00: