search

kubernetes / community

Kubernetes community content
Apache License 2.0
11.97k stars 5.16k forks source link

There should be a kubernetes contributor guide #892

Closed pmorie closed 5 years ago

pmorie commented 7 years ago

I've been working on the service-catalog incubator repo for around the last 11 months. The service-catalog is an extension project shaped similarly to kubernetes from an architectural perspective:

The people participating in the SIG were initially largely unfamiliar with our architectural patterns and conventions. This was a bit jarring for me as a Kubernetes OG - as has been written elsewhere, we have a lot of knowledge transmitted as oral history that isn't written down. One big takeaway from this experience is that it would be extremely useful to have a developer guide for kubernetes and projects like the service-catalog that use the same patterns and have a goal of becoming first class projects under the kubernetes github org.

We have fragments of this knowledge written down already, mostly living in the contributors/devel directory in this repo:

... amongst others. However, these pieces are sprinkled in with a bunch of other stuff that lives in the same directory which is very specific to certain niche topics.

These pieces also don't tell all of the story. For example, the API conventions document doesn't address certain aspects of API design doctrine like how to differentiate behavioral modes for resources (see #859).

In our experience so far in the catalog incubator, we have tended to spend a lot of time discussing these topics which have been resolved already in the minds of many senior core contributors.

It would be great to have these pieces brought together into a coherent whole that is maintained as we evolve conventions. Ideally, someone new to the community would be able to read this guide and have a more than minimal grounding in the concepts necessary to understand existing code and a good idea of how to get started thinking 'the kubernetes way'. In addition to making it easier to onboard new community members, it would help conserve time for SIGs forming around new concepts in the community.

I think this cuts across a number of SIGs:

If we have broad interest in creating a guide like this, it might be effective to create a short-term working group to handle bootstrapping the effort of creating the skeleton of the guide from the existing pieces, identifying gaps, and working with the people knowledgeable about those gaps to close them.

Of course, such a document is only useful in the long-term if it is kept up to date, so I think there will be a long-term investment and culture-shift necessary to maintain the dev guide.

Some examples of similar documentation from other projects:

DirectXMan12 commented 7 years ago

I've asserted before that major structural changes to the code should include "developer docs". Things like "how to write a controller" are not immediately clear (especially since at times certain controllers have used legacy patterns, while others have used new patters), and changes to the standard patterns can be unclear to developers less involved in the nitty-gritty parts of the code base (e.g. the change to shared informers).

I'm not suggesting that every nook and cranny of the codebase needs devel docs, but a general "here's the lay of the land, plus the metro system for major cities" would help out new contributors a lot, IMO.

erictune commented 7 years ago

I love this and would like to help.

I think about this as three pieces:

  1. So you want to extend kubernetes? Here are all the ways you can do that, which to use, and more detailed docs for each one. (Aggregated Apiservers, CRDs, Initializers, Flex Volume plugins, Cloud Provider plugins, Operator patterns, etc). This is targeted at devs that that want to tweak kubernetes, but typically don't contribute on github.com/kubernetes/kubernetes. I call this "Top Level SDK Docs". Core Devs might want to reads this. These should be on http://kubernetes.io.

    1. Kubernetes API Conventions. This is mandatory stuff for people developing in github.com/kubernetes/kubernetes, and strongly suggested for people writing API extensions (CRD/Aggregated APIservers). These should be on http://kubernetes.io too.

  2. Kubernetes Core Developer Docs. Much smaller audience than the two previous ones. This is about anything you need to know to develop on github.com/kubernetes/kubernetes and closely affiliated repos. Stuff like generated code and go style and CI/CD and releasing and so on. This can go on http://github.com/kubernetes/community

0xmichalis commented 7 years ago

@kubernetes/sig-contributor-experience-proposals

ncdc commented 7 years ago

I'm interested in collaborating on API related topics:

I would also be happy to share "how I work", which I realize is quite opinionated and is not something we can necessarily prescribe, but I think it could be useful to new contributors.

cc @bradamant3

ncdc commented 7 years ago

@castrojo we might need some assistance organizing this effort

castrojo commented 7 years ago

Love to help when I get back from holiday, in the meantime recommend cc'ing in @kubernetes/sig-docs-maintainers

jayunit100 commented 7 years ago

+1;

0) I would be happy to contribute as well.... Can we put automation in place for this that tests the snippets and API contructs don't bit rot?

1) It seems like dev doc should focus on controllers, extensions, API stuff as the higher priority .

2) Possibly even some exersizes with the sections could be valuable for onboarding new engineers to cloud native places where kube extensions or support is a part of the daily grind.

On Aug 10, 2017, at 8:45 AM, Andy Goldstein notifications@github.com wrote:

@castrojo we might need some assistance organizing this effort

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

alexandercampbell commented 7 years ago

/cc

I love working on onboarding docs of this type. A couple months ago I reworked development.md and I have a 1-page "welcome to Kubernetes" doc under review here https://github.com/kubernetes/community/pull/883. I would gladly contribute to any other onboarding docs.

DirectXMan12 commented 7 years ago

P.S. I'm totally willing to contribute to these docs -- I'm not just here to complain ;-). I'm especially interested in aggregated API servers (I've already written docs in this area in the apiserver-builder repo) and core mechanics.

chenopis commented 7 years ago

We're actually working on adding a user journey portal to kubernetes.io, which should be tighter onboarding paths for different types of users. I think this would fit nicely into that and would love to coordinate w/ you folks who are offering to contribute to the docs. We should discuss this in more detail at the next SIG-Docs meeting.

Andrew (@chenopis)

On Thu, Aug 10, 2017 at 10:47 AM Solly Ross notifications@github.com wrote:

P.S. I'm totally willing to contribute to these docs -- I'm not just here to complain ;-). I'm especially interested in aggregated API servers (I've already written docs in this area in the apiserver-builder repo) and core mechanics.

— You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub https://github.com/kubernetes/community/issues/892#issuecomment-321624247, or mute the thread https://github.com/notifications/unsubscribe-auth/AXpjrY8Ti1Un0czJOKY8GtBB8eixoM_jks5sW0IjgaJpZM4OyXgU .

--

Andrew Chen, Google Inc Tech Writer for Kubernetes chenopis@google.com

pwittrock commented 7 years ago

Count me in.

chenopis commented 7 years ago

/assign @chenopis

Bradamant3 commented 7 years ago

@chenopis not sure where you were planning to start, but I started (locally) what Other Folks call a content audit, with an eye to roughing out IA/a TOC based on current content, proposed revisions, and identified gaps ^^

zacharysarah commented 7 years ago

/assign @zacharysarah

k8s-ci-robot commented 7 years ago

@zacharysarah: GitHub didn't allow me to assign the following users: zacharysarah.

Note that only kubernetes members can be assigned.

In response to [this](https://github.com/kubernetes/community/issues/892#issuecomment-321673236): >/assign @zacharysarah Instructions for interacting with me using PR comments are available [here](https://github.com/kubernetes/community/blob/master/contributors/devel/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
zacharysarah commented 7 years ago

Round 2:

/assign @zacharysarah

MHBauer commented 7 years ago

As a kubernetes outsider, I'm more than willing to gently rant about I've learned about API Servers and code-generation thereof if somebody else wants to write it down or ask me questions in an office hour or something.

MHBauer commented 7 years ago

@ncdc I'm very interested in how other people, especially kube insiders work.

chenopis commented 7 years ago

This is related to https://github.com/kubernetes/kubernetes.github.io/issues/4548.

bgrant0607 commented 7 years ago

This overlaps with #350

bgrant0607 commented 7 years ago

API-machinery-related topics (e.g., generated code, documenting request structure) should be discussed with SIG API Machinery.

cc @kubernetes/sig-api-machinery-feature-requests

bgrant0607 commented 7 years ago

See also #155

bradtopol commented 7 years ago

Paul, this is an awesome idea. How can I help? Is there any content from https://developer.ibm.com/opentech/2017/06/21/tour-kubernetes-source-code-part-one-kubectl-api-server/ that would be worth pulling into this effort?

mhausenblas commented 7 years ago

I'm as well as many other folks here very interested to contribute. Here are a few questions:

Given the many ideas and people expressing interest, maybe a cross-SIG Working Group is the best way forward?

IMO we need (at least) one meeting where everyone interested gets a chance to provide input. This topic is strategically too important to simply assign it to a small group of people based on a ticket and hope for the best. Transparency and open-minded thinking is what we should strive for, if we want to serve our current and future users, that is, Kubernetes developers well.

chenopis commented 7 years ago

@bradtopol @mhausenblas I guess I "own" this project. The main doc is here, and project tracking will be done here. We can discuss this more at the next sig-docs meeting, which is tomorrow 8/29 @ 10:30-11:20 am PDT. Here's the meeting info:

Kubernetes SIG Docs Agenda:

https://docs.google.com/document/d/1Ds87eRiNZeXwRBEbFr6Z7ukjbTow5RQcNZLaSvWWQsE/edit#

Meeting:

Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/678394311 Or iPhone one-tap (US Toll): +16465588656,678394311# or +14086380968,678394311#
Or Telephone: Dial: +1 646 558 8656 (US Toll) or +1 408 638 0968 (US Toll)

Meeting ID: 678 394 311
International numbers available: https://zoom.us/zoomconference?m=zy_s4GywCQIUgexWdq58vma1f9N8F8s8

mhausenblas commented 7 years ago

Thanks for clarifying @chenopis and I'll try to be there, yes.

bradtopol commented 7 years ago

Hi Andrew,   Thanks for reaching out.  I am Beijing right now and unfortunately won't be able to make the 8/29 meeting but will catch up when I get back.   Thanks,   Brad   Brad Topol, Ph.D.IBM Distinguished Engineer Open Technology & Developer AdvocacyOpenStack Foundation Board Member(919) 543-0646Internet: btopol@us.ibm.comAssistant: Kendra Witherspoon (919) 254-0680     ----- Original message -----From: Andrew Chen notifications@github.comTo: kubernetes/community community@noreply.github.comCc: Brad Topol btopol@us.ibm.com, Mention mention@noreply.github.comSubject: Re: [kubernetes/community] There should be a kubernetes contributor guide (#892)Date: Tue, Aug 29, 2017 5:27 AM  @bradtopol @mhausenblas I guess I "own" this project. The main doc is here, and project tracking will be done here. We can discuss this more at the next sig-docs meeting, which is tomorrow 8/29 @ 10:30-11:20 am PDT. Here's the meeting info: Kubernetes SIG Docs Agenda: https://docs.google.com/document/d/1Ds87eRiNZeXwRBEbFr6Z7ukjbTow5RQcNZLaSvWWQsE/edit# Meeting: Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/678394311Or iPhone one-tap (US Toll): +16465588656,678394311# or +14086380968,678394311#Or Telephone: Dial: +1 646 558 8656 (US Toll) or +1 408 638 0968 (US Toll) Meeting ID: 678 394 311International numbers available: https://zoom.us/zoomconference?m=zy_s4GywCQIUgexWdq58vma1f9N8F8s8 —You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.

 

pmorie commented 7 years ago

Unfortunately I won't be able to attend the sig-docs meeting due to a conflict, but I am happy that this effort is being kicked off. Please let me know if you have any questions for me.

chenopis commented 7 years ago

Outcome of SIG Docs meeting:

Specifically regarding creating/organizing content targeted at Kubernetes codebase contributors, I will approach SIG Community and contact SIG API Machinery, SIG Architecture, and SIG Contributor Experience to start a discussion about the governance of this process. @mhausenblas has agreed to help collect materials and requirements for the actual content. Then once everyone is aligned, we will create a User Journey for K8s code contributors to help onboard them.

mhausenblas commented 7 years ago

From 2017-08-31 K8S Community meeting via @bgrant0607 (persona-related):

@chenopis want to link to your persona doc from here as well?

bgrant0607 commented 7 years ago

Some personas:

There are some finer-grain distinctions / journeys / scenarios we could make, also, such as:

bgrant0607 commented 7 years ago

User journey doc also has personas: https://docs.google.com/document/d/1EdQ8acmuKGlzZy1agejLqmB7cLpTRBJKC0JYptJ-ylg/edit#

mhausenblas commented 7 years ago

I'm working on the Kubernetes Contributors Material Inventory now with an ETA of early w/c 2017-09-04. This inventory aims at listing all the known material and can be used to make an informed decision about the next steps (cross-SIG WG, etc.)

CC: @chenopis

bradtopol commented 7 years ago

Hi @chenopis, I also was not able to make the 8/29 sig-docs meeting due to business travel. Was there agreement on us having a weekly meeting for this topic? How can I get started helping out on this effort? Thanks, Brad

chenopis commented 7 years ago

@bradtopol I'm working with Paris, one of the community managers, and we're going to assemble a working group to tackle this. I've put together a list of contacts based on this issue. We will send out some communications next week -- probably after Tuesday.

mhausenblas commented 7 years ago

@chenopis I've completed the first iteration of the Kubernetes Contributors Material Inventory now for review for the group. Note that I won't be able to join this week's SIG Docs meeting but trying to catch up via mail. Any feedback, please directly in the GDocs …

pwittrock commented 7 years ago

Some contributors may find the following guide useful:

https://github.com/kubernetes-incubator/apiserver-builder/blob/master/docs/concepts/api_building_overview.md

Uses the apiserver-builder libraries to walk through Kubernetes API conceptual information

pmorie commented 7 years ago

Some contributors may find the following guide useful:

https://github.com/kubernetes-incubator/apiserver-builder/blob/master/docs/concepts/api_building_overview.md

Those are amazing docs and should definitely be part of a dev guide as well as user-facing guides, :100:

mhausenblas commented 7 years ago

@chenopis any recent developments here I might have missed?

parispittman commented 7 years ago

What is this README doc missing? This is a combo dev and contributor doc. For @chenopis project, we would be splitting much of this into two personas - the developer or the contributor.

Let's get on a video call and discuss further. ContribX SIG should own the contributor piece to this and I can help facilitate. Here's a doodle for your availability. I'll post the link when we have a majority favored date and time.

pmorie commented 7 years ago

@parispittman

What is this README doc missing?

Notables include:

erictune commented 7 years ago

On Mon, Sep 18, 2017 at 9:18 PM, Paul Morie notifications@github.com wrote:

@parispittman https://github.com/parispittman

What is this README doc missing?

Notables include:

  • How to think about the task of extending Kubernetes via the API aggregator

I have written a "Concept Guide" on Extending Kuberetes. It covers how to pick between API Aggregator and CRD. I will clean up and share shortly.

  • How to build an API server
  • How to correctly design an API resource
  • How to write a controller

I plan to write a Concept doc about the concepts you need to understand to write a controller.

  • The 'why' behind design decisions large and small
  • What you should not try to make Kubernetes APIs do

The above mentioned Extensions Concept Guide has some content on how to decide if you should make your API be a Kubernetes API or not.

  • How to think 'the Kubernetes way'
  • Numerous implementation tasks like how to make a field selectable
  • How to write an admission controller
  • How to write an initializer

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kubernetes/community/issues/892#issuecomment-330424823, or mute the thread https://github.com/notifications/unsubscribe-auth/AHuudrqW31Rps2UtRsPQf_x9PzXmJFG0ks5sj0CmgaJpZM4OyXgU .

parispittman commented 7 years ago

@pmorie I'm still tripping between the definition of developer vs contributor; there are grey areas and areas of intersection but both (mostly) different pathways and onboarding. Should we unpack these and keep them somewhere visible? This will be crucial for the user journeys project. Some of the bullets listed could go in dev, contrib, or both. Sounds like @erictune has and/or can create many of those as well.

Let's get on a call (https://beta.doodle.com/poll/54eyv5vzhka8ht4f). I'll track everyone down separately from this issue.

spzala commented 7 years ago

@parispittman IMHO contributor is a top level term - developer is a contributor but a contributor may or may not be a developer. So the contribute guide should apply to every contributor onboarding but developer guide should be of specific interest to dev. Thanks!

erictune commented 7 years ago

In terms of roles in "Crafting User Journeys" how about these role names:

Kubernetes Contributor. Contributes directly to Kubernetes community projects with or without coding --> Needs a high level guide to how the community works, like SIGs and roles.

Kubernetes Developer. Contributes code directly to Kubernetes community projects --> Needs to read all the docs that Kubernetes Project Contributor does, plus things like guides about the commit flow, coding standards, reviews, etc. --> Also may need to read some Ecosystem Contributor guides for parts of the code they will work on.

Ecosystem Contributor --> Not sure if this role is needed. No ideas of what docs for this role.

Ecosystem Developer --> Writes code that is a client of Kubernetes APIs, extensions, plugins, automation on top of Kubernetes, Apps on top, Operators, etc.

On Wed, Sep 20, 2017 at 9:29 AM, Sahdev Zala notifications@github.com wrote:

@parispittman https://github.com/parispittman IMHO contributor is a top level term - developer is a contributor but a contributor may or may not be a developer. So the contribute guide should apply to every contributor onboarding but developer guide should be of specific interest to dev. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kubernetes/community/issues/892#issuecomment-330907435, or mute the thread https://github.com/notifications/unsubscribe-auth/AHuudnx0PxxlTJ1rrew_OSVZRaaaejU5ks5skT1ggaJpZM4OyXgU .

erictune commented 6 years ago

I've added docs about ways to extend Kubernetes. https://github.com/kubernetes/website/pull/6224

And expanded docs on how to pick between CRD and AA. https://github.com/kubernetes/website/pull/6238

parispittman commented 6 years ago

FYI - #6102(issue against the website repo) has been created to migrate a contributor guide there under the persona of 'contributor', an updated dev guide will be nested in there. The raw doc that we are working from is located inside of the issue.

guineveresaenger commented 6 years ago

UPDATE - the issue Paris mentioned in the previous commit now has a starting point and umbrella issue for a contributor's guide linked to it in comments.

fejta-bot commented 6 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

nikhita commented 6 years ago

/remove-lifecycle stale