Closed HumairAK closed 2 years ago
HumairAK
commented
3 years ago @quaid let me know if I've missed anything. If you (or anyone else) disagrees with any of the suggestions, please discuss them here, and I'll amend them, once we're agreed with the action items, we can convert this into a roadmap.
Feel free to also send this out to the mailing list so we can get more eyes/input on it.
HumairAK
commented
3 years ago My thoughts on 1-4:
apps/docs repo, then re-organize on the website how information ought to be displayed. It's much easier to structure a ToC then figure out which markdowns/notebooks need to go to which repos, for both newcomers and existing contributors (myself included).apps/docs accordingly. I think this solves multiple problems:
But we need some tooling work to create an editorial workflow so we can scale documentation. This is because the docs are embedded in various repos as MD and I guess IPYNB files across a growing number of repos. That is the model, correct, having docs directly in each repo? That 5's fine but we need to centralize the editorial review and publish workflow if you want docs to scale beyond just you all.
HumairAK
commented
3 years ago Some of the suggestions from above in this crappy illustration:
Edit icon on the top right inspired by k8s
quaid
commented
3 years ago There has been discussion on slack regarding the state of the docs. Recently @quaid pointed out various issues, anti-patterns w.r.t community/contributor growth. The full discussion is linked at the end of this post for context. In this issue I shall try to address what I think are the main points with possible solutions. My hope is that we can create some actionable items that can put us on a positive trajectory and help establish practices that will keep all our docs clear, contributor friendly, and scalable.
Thanks, this is a good way to start. I'll get my acks/nacks, replies, and thoughts included, then send an email to the mailing list about the activity.
[ ] 1. Figure out if we need HitchHiker's Guide (HtG), Community Handbook, and Docs to be in separate repos.
- Can they be merged? If yes where do they live?
- If they can't, we need very clear guidelines on the purpose of each, what goes where, this should be stated in the repo Readme on the front of the repo.
From an open source project perspective, there are two kinds of documentation: intended for users of the project's artifacts/services/releases; intended for contributors to the project itself.
Something called e.g. "devel-docs" is without contextual meaning—is it for users of the project who are developing on/with project's software? or is it for developers of the open source itself?
In the case of Operate First, we have an usual situation: the content for the user and contributor docs is the exact same, in many cases.
But that's just content, it's not reflective of how a document is focused.
For example, a contributor task might be onboarding OSS Project Foo and it's going to require Kafka, Loki, and Grafana, as well as a database Operator. Then the task in the "Community Handbook" (contributor-focused docs) is called, "How to onboard a new open source software project as a full-service tenant," and the handbook page has sub-sections for various possible needs. One sub-section pulls in the Kafka overview and then renders the rest of the pages in /apps/docs/odh/kafka/ as steps in the how-to.
By comparison, a user such as an SRE learner would have a learning module that pulled the same how-to content in but presented it steps in a learning path. They would soon reach a step in that journey that pulled up Jupyter Notebooks, including the ability to open and explore a cloud-native learning environment. If they find a bug or think of a feature/fix to offer, there can be a step link that user to the section in the Community Handbook on "How-to suggest a change to the website."
From that, what I see is that the the HtG content is contributor-focused (Community Handbook), but needs a new-contributor how-to face put in front of it (which ends up supporting all contributors.) It can also help form part of the SRE learning pathway, I think; or an SRE-contributor learning path.
- [ ] 2. If HtG stays, rename it to something else that doesn't already exist and will not result in concerns over copyright.
I'm not sure about the rights concern, the main thing there is this is an important step when naming things, and in this case it might or might not mater.
The name has a bit of an insider feel to it, I think, and also isn't descriptive of the contents even for people who understand the literary reference.
- [ ] 3. The content currently under HtG on the website should be renamed "Community Handbook"
Yes, and we want to go through and organize the Table of Contents of the Community Handbook to be task oriented, and begin to write short to medium how-to articles that instead point to or pull in these various pages. (This is if I'm reading the content correctly.)
- [ ] 4. Notebooks are a higher learning curve and not beginner-friendly, separate these out as for more "advanced" users, who are interested in automating tasks. There should also be clear guides on how to run these Notebooks locally without overhead (like instructions on installing jupyterlab, other tools, etc.)
+1
It's also great if the advanced notebooks are built from the same base components as the rest of the documentation, so the how-to from the Community Handbook are the actual steps or formula in the Notebook. Super extra cool bonus space points if we can literally use the same Markdown files.
- [ ] 5. We need some tooling work to create an editorial workflow (centralized) so we can scale documentation.
This also resolves the "where it lives" question, doesn't it?
- [ ] 6. Documentation that lays out a learning path for SRE's. Suggestion: Katacoda tutorials for knowing openshift/k8s, github ramp up, gitops, our instructional videos, etc..
Most of the general SRE learning paths are being developed by a Red Hat team who are just getting started around the edges in GitHub and on the mailing list: Stephanie, Sarvesh, Parth, Yogita, Moira, and others.
They have recently seen the presentation by Jonathan Appavoo and are interested in all of the above with Jupyter Notebooks. So it's possible we'll deliver the SRE training that way, but built by curriculum developers and with the support so it's not an Advanced experience.
For the Katacoda part, we're talking with a number of additional folks about the possibility of spinning up a new cloud-native interactive learning environment. Folks who are looking to migrate away from those type of solutions, there isn't (yet) a cloud native way to do this. So let's put a hold on this part and watch/participate in the other activities as a way to fill this part.
- [ ] 7. Replace/rename "contribute to this page" button with an "edit this page" button, that will link to the markdown file (same as the current "contribute to this page" button). And reword docs so that each page starts with a set of "Pre-requisite tools and knowledge" required in order to understand the document in question. Knowledge portion should link to other docs that help the reader gain the necessary domain knowledge to work through the current doc.
Thank you, yes. We can even include another link next to the "edit this page" (or on each page itself in some way?) a link to the Community Handbook page on "How to edit a page on the website."
- [ ] 8. If we have multiple locations with docs, then each repo should have the same set of approvers/reviewers that have a clear understanding of what the roadmap/direction is for our handbooks/guides/docs and so on.
- [ ] 9. We should also ensure all relevant parties are added to reviewers/approvers list for the website, and that there is clear understanding/consensus by the approvers on the roadmap for website organization.
I think we may want to break out the discussion or at least create a draft file to hack on together around the locations of docs.
A single Markdown file can, in my view, be part of i) contributor docs, ii) user docs, iii) user training pathways, and iv) contributor training pathways.
So it may not matter if the files are embedded within the various service repos. The key is to label each of the Markdown files with one or more of the label groupings (since it can be in one or more ToCs/modules):
area/contributor and kind/handbook, then it's a Community Handbook page and should be findable via the ToCarea/user and kind/documentation, then it's end-user documentationquaid
commented
3 years ago My thoughts on 1-4:
- Put everything in
apps/docsrepo, then re-organize on the website how information ought to be displayed. It's much easier to structure a ToC then figure out which markdowns/notebooks need to go to which repos, for both newcomers and existing contributors (myself included).
I think this is going in the right direction, and to be clear, I do see reasons for having a flat space with all the content within it (in one directory, sub-directories, etc.)
But you have tapped something here that is kind-of another category of documentation than user-focused and contributor-focused, and that is reference documentation.
It may be useful to have all the docs specific to an app in a folder for just that app. The reason is simply the reason the current HtG layout is useful: a list of apps full of reference answers is helpful to people for innumerable reasons.
That could pair with a different structure that has the how-to/task-oriented documentation being all in /community-handbook/contributor-area/
- Get rid of Hitchhiker's guide entirely, have a separate section for "automated docs" (with a better/intuitive name) on the website that will link to a Jupyterbook with only notebooks. This is for users experienced with notebooks, but we also provide instructions on how one can set up their own jupyterlab/environment and use these notebooks if they are interested in learning.
I think this fits into the overall picture; especially because my view is all the docs could be/should be automated.
- Jupyterbook + notebooks (i.e. current HTG) contents go in the apps repo, so that we can update notebooks alongside other PRs, similar to markdowns. If we need to have a separate repo for build pipelines for the juptyerbook, then we can have a sync job that can move the contents to a new repo, and have this repo only exist for build pipelines (no one interacts with it directly).
If I understand this correctly, this may sounds like several of the steps of a single-source content approach.
To be clear, the curriculum development team are planning hands-on to help create the Markdown file structure, collaborate with us the automagic we do there. Now that @stefewrite et al have seen the possibility, we can all discuss doing this with Jupyter, and figure out how we want to do the automagic for swapping/injecting/tracking multiple MD files across multiple content builds.
- Retire community-handbook repo, move contents to
apps/docsaccordingly.
Aside from the categorization differences I pointed to for app reference vs end-user vs contributor vs learning (user|contributor), there is a concept behind the Community Handbook that benefits from adopting the same name as other instances of community handbooks.
There is a full explanation and pitch I made on the mailing list mid-September, here are the key points:
I think this solves multiple problems:
- Having them in one location is a lot more newcomer friendly, knowing there's one place to look for everything is demonstrably easier then having to guess and go searching
Generally, yes. Practically "in one location" may translate to "associated with an app or article in the Community Handbook".
I am leaving out the possibility that we'll want a "User Handbook" that is all the how-to/what-is content built from all the other bits, but organized and focused for specific user needs.
- One location means, you're more likely to catch overlapping content, and enforce more consistency amongst your docs
- Having to watch one repo, means you are more likely to have more eyes on your PRs (less repos to watch -> more people congregate to the one repo)
This I fully get, I miss quite a bit still.
Ultimately, though, I think this speaks to having tooling that goes one layer above the repo watching level. For us to turn regular end users into contributors (one of the main techniques for growing a contributor base), we need to lower the barriers to contributing. For example, we may have an experienced developer on an OSS project who knows about how to watch code being developed via GitHub notifications, may not be able to apply that same expertise to watching a documentation repo. And we certainly know that by-definition the vast majority of possible contributors know little to nothing about using GitHub.
So we can add some automation to existing automated columns in the Community Handbook project board perhaps to take advantage of the labels and so on. I have found writers and editors who haven't used GitHub can more easily understand how to use an editorial board like this than to understand an issue or PR from scratch as a stand-alone thing.
You can peruse the editorial board we use for the Open Source Way to see how that has worked for writing longer chapters, and what that looks like in terms of the repo flow. (As an example, ideas and ongoing chapter writing projects are associated 1:1 with an issue (and the content as associated PRs), and those can be in stasis for months while content is being developed outside of GitHub. Our current Op1st auomation (Sesheta) goes through and marks issues as stale after 90 days; we wouldn't want to do that the same way in an editorial board.
- It's easier to scale as it addresses this point made by @quaid :
But we need some tooling work to create an editorial workflow so we can scale documentation. This is because the docs are embedded in various repos as MD and I guess IPYNB files across a growing number of repos. That is the model, correct, having docs directly in each repo? That 5's fine but we need to centralize the editorial review and publish workflow if you want docs to scale beyond just you all.
- We continue to have the freedom to re-organize markdowns into different tabs/categories on the website so there's no loss there.
Right, of course, we can consider the website itself as a build artifact for this single-source of documentation. So far we have:
Write once, automatically rebuild into content everywhere!
4n4nd
commented
3 years ago How about we start with moving all the docs to one place (apps repo) and start tagging them like @quaid suggested?
HumairAK
commented
3 years ago Okay so the low hanging action items I see right now that are not being contested are:
Does anyone disagree?
Most of the general SRE learning paths are being developed by a Red Hat team who are just getting started around the edges in GitHub and on the mailing list: Stephanie, Sarvesh, Parth, Yogita, Moira, and others.
Do we have any sort of time frame for this? Where can we follow these discussions?
Also looking for clarifications on:
For example, a contributor task might be onboarding OSS Project Foo and it's going to require Kafka, Loki, and Grafana, as well as a database Operator. Then the task in the "Community Handbook" (contributor-focused docs) is called, "How to onboard a new open source software project as a full-service tenant," and the handbook page has sub-sections for various possible needs. One sub-section pulls in the Kafka overview and then renders the rest of the pages in
/apps/docs/odh/kafka/as steps in the how-to.By comparison, a user such as an SRE learner would have a learning module that pulled the same how-to content in but presented it steps in a learning path. They would soon reach a step in that journey that pulled up Jupyter Notebooks, including the ability to open and explore a cloud-native learning environment. If they find a bug or think of a feature/fix to offer, there can be a step link that user to the section in the Community Handbook on "How-to suggest a change to the website."
and:
Right, of course, we can consider the website itself as a build artifact for this single-source of documentation. So far we have:
End-user task-oriented docs
End-user training content
Anyone-curious reference content
Contributor task-oriented docs
Contributor training content
Advanced contributor docs (stripped-down Notebooks)
Website itself
I'm not following, to me both personas listed above seem like contributors. In my mind a user was someone who is not making PRs. So using the managed services, the platform, deploying things to a namespace, and so on. An SRE learner is someone who's aiming to help with SRE tasks and thus eventually make PRs so they would fall under the contributor scope. Let me know what I'm missing.
Also it was my understanding that our intended goal is that everyone participates in contributing. As soon as someone requires some operations related task, we ought to have docs available so that they may do it themselves. If we don't, we then need to add them.
Therefore, I'm a little confused why we need the distinction between user/contributor docs (thus doubling the effort). Why not just have contributor docs, that list pre-requisite knowledge, and links on where to attain them before attempting the doc? The pathways can just be putting them in a logical order.
Also don't understand why we would distinguish reference docs from contributor docs. In this case they seem to be essentially the same. In Software, there is a clear distinction, as you have ref docs for cataloging programming elements, technologies, and contribution docs for those wanting to change these elements. But I'm not sure if there's an equivalent and clear distinction in open source operations/gitops. How-To Docs for us just usually mean "make a pr changing this x set of configurations to get y result."
there is a concept behind the Community Handbook that benefits from adopting the same name as other instances of community handbooks.
Sorry - to be clear, I'm not suggesting to get rid of the handbook, just to move its contents withing apps/docs so it can be managed/maintained in the same location as the rest. The repo and its contents can exist as apps/docs/community-handbook/* . Here I'm using the words docs/documentation as an all encompassing term for all written content (ref docs, guides, books, contributor docs, what have you) if there's a better term for this to avoid confusion let me know.
Ultimately, though, I think this speaks to having tooling that goes one layer above the repo watching level. For us to turn regular end users into contributors (one of the main techniques for growing a contributor base), we need to lower the barriers to contributing. For example, we may have an experienced developer on an OSS project who knows about how to watch code being developed via GitHub notifications, may not be able to apply that same expertise to watching a documentation repo. And we certainly know that by-definition the vast majority of possible contributors know little to nothing about using GitHub.
So we can add some automation to existing automated columns in the Community Handbook project board perhaps to take advantage of the labels and so on. I have found writers and editors who haven't used GitHub can more easily understand how to use an editorial board like this than to understand an issue or PR from scratch as a stand-alone thing.
You can peruse the editorial board we use for the Open Source Way to see how that has worked for writing longer chapters, and what that looks like in terms of the repo flow. (As an example, ideas and ongoing chapter writing projects are associated 1:1 with an issue (and the content as associated PRs), and those can be in stasis for months while content is being developed outside of GitHub. Our current Op1st auomation (Sesheta) goes through and marks issues as stale after 90 days; we wouldn't want to do that the same way in an editorial board.
None of this seems to conflict with the notion of moving everything to one repo. Project boards are not dependent on a repo. You can still have project boards for editorial purposes.
schwesig
commented
3 years ago
- [ ] 1. Figure out if we need HitchHiker's Guide (HtG), [..]
+1 for merging | +1 for using labels to separate
- [ ] 2. If HtG stays, rename it [..] that doesn't already exist [..] concerns over copyright.
if we go with labels, renaming is not an issue | but it shouldn't be an issue at all https://github.com/operate-first/hitchhikers-guide/issues/9
- [ ] 3. The content [..] under HtG [..] be renamed "Community Handbook"
+1 | labels etc see 1. and 2.
- [ ] 4. Notebooks are a higher learning curve [..], separate these out as for more "advanced" users [..] guides on how to run these Notebooks locally without overhead [..]
-1 separate these out --> all should be as much automated as possible, following @quaid, but labelled? | +1 guides & local use & w/o overhead
- [ ] 5. We need some tooling work to create an editorial workflow [..] we can scale documentation.
+1 | thinking modular, a doc/md needs to be as small as possible/focussed on a solution, that non part of a doc is redundant somewhere else, and the meta and submeta toc comines them (doc follows idea of container/microservices; just one small purpose and orchestrate it)
- [ ] 6. Documentation that lays out a learning path for [..]
+1 learning path | try to catch all different kind of people/mindsets/searches/approaches/goals *1
- [ ] 7. Replace/rename "contribute to this page" button with an "edit this page" button, [..] each page starts with a set of "Pre-requisite tools and knowledge" [..] link to other docs that help the reader [..] to work through the current doc.
+1
- [ ] 8. If we have multiple locations [..] each repo should have the same set of approvers/reviewers [..]
IF we have multiple locations THEN +1 for same approvers (I am +1 less repositories)
- [ ] 9. We should also ensure all relevant parties are added to reviewers/approvers list for the website, and that there is clear understanding/consensus by the approvers on the roadmap for website organization.
not really an opinion on that | but +1 for clear consensus
--footnotes-- *1 we don't know, what kind of problems, questions, goals and knowledge visitors/members will have. therefore most likely we will not create the correct names/headlines for the paths. therefore maybe solutions can be:
mho on different types/mindset:
some people are looking for a product, e.g. "I need a Porsche, no matter what it can or does, but everybody is talking about it." some people are looking for features, e.g. "My car needs 300HP and an excellent water cooled brake system." some people are looking for benefits, e.g. "I need to get from work to home really fast, and my car needs to stop as fast and safe as possible." and when it comes to learning paths... these can be in all areas: from wanting to learn about a Porsche, over how can I tune and optimize a motor, to how to drive it in the best way.
quaid
commented
3 years ago +1 | thinking modular, a doc/md needs to be as small as possible/focussed on a solution, that non part of a doc is redundant somewhere else, and the meta and submeta toc comines them (doc follows idea of container/microservices; just one small purpose and orchestrate it)
ding, ding
That is a great way to describe docs in this context, I am formally adopting that simile.
--footnotes-- *1 we don't know, what kind of problems, questions, goals and knowledge visitors/members will have. therefore most likely we will not create the correct names/headlines for the paths.
Yes and no. It's true we mainly won't know what open source developers want, but we know our Open Source Developer persona needs to have self guided access to solutions-tagged documentation and training. And we know that our SRE persona and our Data Scientist and Data Engineer personas need to have a specific learning pathways related to those skill sets. So that gets us going for the user side of documentation. Where it comes to contributor-side the general idea is to meet in the middle where our current contributor documentation is and where we want the user documentation to be so there's a smooth transition from user to contributor in process, docs, training, and so forth.
schwesig
commented
2 years ago That is a great way to describe docs in this context, I am formally adopting that simile.
it would be my honor if you are adopting it
we don't know, what kind of problems, questions, goals and knowledge visitors/members will have. therefore most likely we will not create the correct names/headlines for the paths.
[..] but we know our Open Source Developer persona [..] need to have a specific learning pathways related to those skill sets. [..] general idea is to meet in the middle [..] so there's a smooth transition from user to contributor in process, docs, training, and so forth.
agreed! my introduction "we don't know" was not meant as a locked-down "we will never know". I used it to get to my point, that there should be more than one way to create or find a path. Find Path --> Search; Create Path --> (dynamically) orchestrated doc-modules | both as mentioned above
HumairAK
commented
2 years ago +1 for merging | +1 for using labels to separate
@schwesig what do you mean by labels in this context? what does that look like in implementation?
HumairAK
commented
2 years ago @quaid any thoughts on this comment: https://github.com/operate-first/SRE/issues/424#issuecomment-966433198
once again I'd like to reach some sort of consensus in these areas and start thinking implementation, as we have dedicated a portion of this sprint time to address some of these concerns.
4n4nd
commented
2 years ago 1. End-user task-oriented docs
2. End-user training content
3. Anyone-curious reference content
4. Contributor task-oriented docs
5. Contributor training content
6. Advanced contributor docs (stripped-down Notebooks)
7. Website itself@quaid can you explain where each of these items would go? As in which repo the docs for each item will be stored in?
schwesig
commented
2 years ago [..] +1 for using labels to separate
@schwesig what do you mean by labels in this context? what does that look like in implementation?
@HumairAK if we, for reasons like clarity/easy to navigate/building paths/etc, still need the separation between HtG, Community Handbooks (CoHb), and Docs (, and more?), we could add labels for that. e.g. /book Htg or /book CoHb
HumairAK
commented
2 years ago Sorry...still not following, how are we labeling the content? like, if I have a how_to_x.md how are we labelling it so we can find this content? /book htg what does this mean? are you talking about how we label gh labels like /kind etc.? but that's for issues, what does this look like on a markdown and on a site?
schwesig
commented
2 years ago Sorry...still not following, how are we labeling the content? [..]
I was referring to the labels and ideas you and @quaid were mentioning and referring to above. I don't necessarily think we need that, I was just was adding my ideas to it. If I was getting a wrong idea how it was meant, just ignore. Maybe replace label with (meta)tags. /book htg was meant as an example to mark an entry/doc as formerly part of the Hitchhikers Guide.
just an idea that just came into my mind: if all solutions/docs/mds should be based on one issue. If we follow the modular way, this could become: one issue --> one solution --> one doc/md --> (optional) 1 to x PRs
quaid
commented
2 years ago Okay so the low hanging action items I see right now that are not being contested are:
- Change contribute button to edit button, and have a new button for "How do edit a page on the website."
+1, and maybe it would be OK if the edit button said, "Edit this page on GitHub", so it's clear what is happening before the link is clicked. Having an inline edit link makes maintaining pages easier.
The "How to edit..." button is a good idea. It's going to start looking crowded soon, though; perhaps a design element can keep it from looking like a mess of buttons.
- Retire HtG repo, move notebooks to apps repo, and on the website link to it, create pipeline to copy contents to another repo for build-pipeline workflow (publishing Jupyterbook so that it maybe linked from the website as shown in the sample illustration)
OK, iiuc this puts notebooks directly with the apps they are documenting, yes?
I think it makes sense to keep docs with the thing they are documenting, and centralize the editing and reading process, as we've been discussing.
- Rename HtG on website to "Community Handbook"
Well, yes and ... immediately redo the table of contents?
No, wait ... I think the challenge is the HtG is narrowly-focused but fairly complete within that focus, whereas the CHbk is broadly focused and benefits from organic contributions.
We want a big, empty, write-articles-here Table of Contents for the Community Handbook. And the HtG content fits within that as far more complete area.
Does anyone disagree?
Most of the general SRE learning paths are being developed by a Red Hat team who are just getting started around the edges in GitHub and on the mailing list: Stephanie, Sarvesh, Parth, Yogita, Moira, and others.
Do we have any sort of time frame for this? Where can we follow these discussions?
Time frame is ongoing, we're working with the team on the tools and practices of this community. In fact, this morning I spent an hour with some of them talking about repo structure, which resulted in the mailing thread I started today.
Also looking for clarifications on:
For example, a contributor task might be onboarding OSS Project Foo and it's going to require Kafka, Loki, and Grafana, as well as a database Operator. Then the task in the "Community Handbook" (contributor-focused docs) is called, "How to onboard a new open source software project as a full-service tenant," and the handbook page has sub-sections for various possible needs. One sub-section pulls in the Kafka overview and then renders the rest of the pages in
/apps/docs/odh/kafka/as steps in the how-to.By comparison, a user such as an SRE learner would have a learning module that pulled the same how-to content in but presented it steps in a learning path. They would soon reach a step in that journey that pulled up Jupyter Notebooks, including the ability to open and explore a cloud-native learning environment. If they find a bug or think of a feature/fix to offer, there can be a step link that user to the section in the Community Handbook on "How-to suggest a change to the website."
and:
Right, of course, we can consider the website itself as a build artifact for this single-source of documentation. So far we have:
End-user task-oriented docs End-user training content Anyone-curious reference content Contributor task-oriented docs Contributor training content Advanced contributor docs (stripped-down Notebooks) Website itselfI'm not following, to me both personas listed above seem like contributors. In my mind a user was someone who is not making PRs. So using the managed services, the platform, deploying things to a namespace, and so on. An SRE learner is someone who's aiming to help with SRE tasks and thus eventually make PRs so they would fall under the contributor scope. Let me know what I'm missing.
OK, I understand what you mean from a pedantic point of view: we have a contribution mechanism (the pull request), so that must signal one is a contributor.
However, it leaves out the contributor who is not working via PR. A lot of open source contribution happens outside of the git workflow.
What's I believe is important about thinking of a "user" as a unique persona is to helps make sure we are casting our net as wide as possible, that is, you don't have a description that is too narrow that it leaves out casual readers, tech journalists, curious corporate developers, and so forth.
About the SRE Learner, the group you describe are using what I labeled as "Contributor training content": they are contributors training on SRE content to work on operations.
The SRE Learner is a user persona who's interest is in learning about SRE practices, taking coursework, and so forth. The content and training for these users is going to be created in the open by a growing community of practice around SRE. Most likely this user is going to outnumber the contributor learners by many orders of magnitude.
One key idea is around "shifting left" on the ability to contribute. Eventually, a user training on SRE practices should be able to take their scratch training space and promote it to PR-path with a single click.
Also it was my understanding that our intended goal is that everyone participates in contributing. As soon as someone requires some operations related task, we ought to have docs available so that they may do it themselves. If we don't, we then need to add them.
This sentence sounds like you are using the definition of contribution to mean something done using gitops. A contribution is much more than that. And if you mean that all the types of contributions in the list below plus others should have a way to be expressed and tracked using git, that can be an aspiration for the project but that's not where we are at.
https://www.theopensourceway.org/the_open_source_way-guidebook-2.0.html#_what_is_a_contribution
This article is another way to look at contribution in the context of a project:
https://www.redhat.com/en/blog/how-your-career-goals-can-focus-your-open-source-contributions
Therefore, I'm a little confused why we need the distinction between user/contributor docs (thus doubling the effort). Why not just have contributor docs, that list pre-requisite knowledge, and links on where to attain them before attempting the doc? The pathways can just be putting them in a logical order.
I believe the categories "Contributor task-oriented docs", "Contributor training content", and "Advanced contributor docs (stripped-down Notebooks)" are the only ones related to what you are describing.
Within those categories, there is a functional difference between task-oriented SOPs and a training module. The underlying content can and should be the same, but the format and UX are different. Finally, I understand there is an appetite for a "quick key to just get my work done" idea behind the way the HtG and its notebooks are laid out, and that's the "(stripped-down Notebooks)".
Also don't understand why we would distinguish reference docs from contributor docs. In this case they seem to be essentially the same. In Software, there is a clear distinction, as you have ref docs for cataloging programming elements, technologies, and contribution docs for those wanting to change these elements. But I'm not sure if there's an equivalent and clear distinction in open source operations/gitops. How-To Docs for us just usually mean "make a pr changing this
xset of configurations to getyresult."
"I'm an Open Source Developer just trying to get my software ready for onboarding, and I don't need my hand-held in any way. I want the schematics and references, so I can make my own learning pah."
Maybe it doesn't make sense in the end or it's too hard to do, but my experience suggested to me it could be useful content to create, especially if it can be automated out of the other material.
there is a concept behind the Community Handbook that benefits from adopting the same name as other instances of community handbooks.
Sorry - to be clear, I'm not suggesting to get rid of the handbook, just to move its contents withing
apps/docsso it can be managed/maintained in the same location as the rest. The repo and its contents can exist asapps/docs/community-handbook/*. Here I'm using the wordsdocs/documentationas an all encompassing term for all written content (ref docs, guides, books, contributor docs, what have you) if there's a better term for this to avoid confusion let me know.
Well, it still has to do with branding and findability, and is rooted in our different uses for the word "contribution".
/operate-first/community-handbook is I believe far more discoverable than /apps/docs/./apps/. It shouldn't be subject to the more stringent access controls for who can act as a reviewers or hands-on editors, for example.Ultimately, though, I think this speaks to having tooling that goes one layer above the repo watching level. For us to turn regular end users into contributors (one of the main techniques for growing a contributor base), we need to lower the barriers to contributing. For example, we may have an experienced developer on an OSS project who knows about how to watch code being developed via GitHub notifications, may not be able to apply that same expertise to watching a documentation repo. And we certainly know that by-definition the vast majority of possible contributors know little to nothing about using GitHub. So we can add some automation to existing automated columns in the Community Handbook project board perhaps to take advantage of the labels and so on. I have found writers and editors who haven't used GitHub can more easily understand how to use an editorial board like this than to understand an issue or PR from scratch as a stand-alone thing. You can peruse the editorial board we use for the Open Source Way to see how that has worked for writing longer chapters, and what that looks like in terms of the repo flow. (As an example, ideas and ongoing chapter writing projects are associated 1:1 with an issue (and the content as associated PRs), and those can be in stasis for months while content is being developed outside of GitHub. Our current Op1st auomation (Sesheta) goes through and marks issues as stale after 90 days; we wouldn't want to do that the same way in an editorial board.
None of this seems to conflict with the notion of moving everything to one repo. Project boards are not dependent on a repo. You can still have project boards for editorial purposes.
I think the problem is we agree docs related to a managed service should live close to that service in /apps/docs/, but that leaves out all the contributor-oriented docs not related to a managed service.
quaid
commented
2 years ago @quaid can you explain where each of these items would go? As in which repo the docs for each item will be stored in?
These are the build artifacts, not (necessarily) the repos. It might be a 1:1 or different relationship. This thread on the mailing list points to this discussion on number and type of repos.
Here are some guesses where the content that comprise these builds (the source content) might come from:
End-user task-oriented docs
End-user training content
- SRE Community of Practice knowledge pool
- Live production SRE files
Anyone-curious reference content
To me this is what the current iteration of the Hitchhiker's Guide most resembles. It could be generated out of the Notebooks?
Contributor task-oriented docs
Contributor training content
Advanced contributor docs (stripped-down Notebooks)
- SRE Community of Practice knowledge pool
- Live production SRE files
Website itself
- SRE Community of Practice knowledge pool
- Live production SRE files
- Other stuff
quaid
commented
2 years ago Sorry...still not following, how are we labeling the content? like, if I have a
how_to_x.mdhow are we labelling it so we can find this content?/book htgwhat does this mean? are you talking about how we label gh labels like/kind etc.? but that's for issues, what does this look like on a markdown and on a site?
To avoid having each Markdown file carry metadata in a comment block, I think this may work:
The Markdown file in a git repo is not a thing, it is a representation of the state of that Markdown file at the moment of commit. That commit (PR) can have labels. So if the labels are hooks at the moment of commit, a bot can go rebuild all the various content locations labeled-for. It builds them to locations such as operate-first.cloud/community-handbook or o-f.c/SRE, and so forth.
Can you see something like that working as a trigger and guide for automation?
schwesig
commented
2 years ago
- The Community Handbook should have a low barrier to using it and contributing to it. [..] So a repo called
/operate-first/community-handbookis I believe far more discoverable than/apps/docs/.
+1 Really good point, haven't thought about that. Absolutely valid.
HumairAK
commented
2 years ago @quaid okay I think I have a better understanding of how we're defining contributor/user here. Being mostly operations focused, I'm often writing technical docs that showcase how to interact, onboard, and contribute (direclty to operations). I would like to ensure that there is a clear location where we can organize these particular docs, so that it doesn't disrupt the work being done elsewhere. For clarity I'll refer to these as ops-contributor docs, so as not to be confused with the broader definition of contributors. Everything in apps/docs fall under these ops-contributor docs.
HtG is also ops related, and cover content directly related to ops & making PR contributions to the apps repo specifically, so they ought to be located next to each other.
Now based on your explanation, it seems like these ops-contributor docs should be part of the community-handbook, as a sub-part alongside everything else contributor related. We need these docs on the website so we can link them to folks when they ask us how to do something, so we need an immediate solution to this that's least disruptive to other Community-handbook related efforts.
We want a big, empty, write-articles-here Table of Contents for the Community Handbook. And the HtG content fits within that as far more complete area.
Suggestion:
GitOps Docs in this diagram links to a separate Jupyterbook that contains all our apps/docs + HtG (to be moved to apps/docs).
Now when we need to add any new documentation gitops related, it will be de-coupled from all other efforts so as not to disrupt progress there, whilst remaining part of the Community Handbook.
WDYT?
quaid
commented
2 years ago Now when we need to add any new documentation gitops related, it will be de-coupled from all other efforts so as not to disrupt progress there, whilst remaining part of the Community Handbook.
WDYT?
@HumairAK by gum, I think you've got it!
Indeed, I think this is what we want to drive for. Having specific types of docs live near the application/action, and have those as a sub-part of the overall Community Handbook. In fact, GitOps Docs is a good label from what I called "reference docs".
This is the vision for how those GitOps contributor-ops docs interacts with the two kinds of SRE training:
area/user) is following a course module called "How GitOps Works". The content source is drawn from the SRE knowledge pool, then crafted into the course module, which is delivered as a Jupyter Notebook rendered into course pages.ops-contributor content, pulled from /apps/docs/, and used as an example. So when the switch over to JupyterHub happens, the interactive learning environment is using actual ops-contributor content and tools from /apps/ to do gitops work in the hands-on tutorial.Now imagine a person who is interesting contributing to operations in the community; they could even be the above learner having just reached the moment of learning about GitOps and they have a simple contribution they want to make. It may even be a contribution they have right there in JupyterHub, so they click on a "Turn this idea into a contribution" button. If it's an existing or incoming contributor looking around, they'll find themselves in the Community Handbook:
recipes becomes shopping lists and recipes.Make this sandbox into a real pull request. An actual PR process is started, with the patch for splitting the directories as the starting point.And then of course, in the first scenario, if an SRE Learner suddenly gets an idea for a contribution, the Make this sandbox into a PR button is right there for them.
HumairAK
commented
2 years ago Having specific types of docs live near the application/action, and have those as a sub-part of the overall Community Handbook. In fact, GitOps Docs is a good label from what I called "reference docs".
Okay great, so I'll be setting up a separate Jupyterbook that contains these gitops docs, for us to categorize our markdowns and notebooks. The ops team will continue to update their documentation from here, and it should not conflict with other work. This book will be linked to by the website as shown in the diagram above (I didn't see any objections here) under the community handbook.
In the future, any new learning modules or whatever can leverage these notebooks/markdowns in their separate jupyterbooks that are structured as courses and whatever else.
sesheta
commented
2 years ago Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
/lifecycle stale
HumairAK
commented
2 years ago I think the gitops docs jupyterbookserves our purposes for the sig-operations side. Moving forward future decisions/discussions regarding these matters will be delegated to the contrib/ops docs groups.
There has been discussion on slack regarding the state of the docs. Recently @quaid pointed out various issues, anti-patterns w.r.t community/contributor growth. The full discussion is linked at the end of this post for context. In this issue I shall try to address what I think are the main points with possible solutions. My hope is that we can create some actionable items that can put us on a positive trajectory and help establish practices that will keep all our docs clear, contributor friendly, and scalable.
Slack Discussion
> **Karsten (quaid) Wade wrote:** > Starting a thread regarding the `Community Handbook` and the `hitchhiker's guide` and tagging Marcel Hild Tom Coufal Humair Khan Anand Sanmukhani Michal Drla. We need to come to an agreement and make a decision, which ultimately needs to happen on the mailing list, but we've also got a timely problem to see if we can solve by Tuesday morning about 09:30 PST when Marcel and I have a lightning talk go live in front of IBM SRE folks and we wanted to have the correct documentation available. We also need to have a frank discussion about why we have a problem, and I think I'd like the frankness here at this point instead of the mailing list. > > **Karsten (quaid) Wade wrote:** > OK, frankly this is a challenge because I've tried to head this off a few times and get support going for the Community Handbook. Let me see if I can drop the backstory bits and focus on the problem. > > OK, I wrote it all out below, it's complex, here is the best tl:dr and then more details beyond: > 1. The `hitchhiker's guide` seems to be in conflict with the `Community Handbook` in terms of purpose, positioning on the website, and attention of all y'all. > 2. The current iteration of the HtG doesn't to do what it says it does. > a. It's presented as a flat list of applications instead of focusing on "how to perform contributor-related tasks". You have to guide yourself around to find procedural pages. > b. The link `Contribute to this page` in this case is spraying anti-contributor repellent all over the page. Every link just dumps you to the MD file, with no guidance on what to do to actually contribute to the page. > c. The entire experience presumes a huge amount of existing knowledge behind that button, and shows no way to get to that knowledge, doesn't tell you that you need the knowledge, so you have to know that you don't know what you don't know! And of course you then have to go figure it all out yourself, then come back to `Contribute to this page`. > d. It does all this while claiming to encourage "_input and contributions from the community_". There is honestly very little that is encouraging about the `Contribute to this page` link outside of the very small set of people in the world who already know how to contribute to a GitHub-based project. We don't need to start by competing for their attention. We need to make it accessible for the other 2 or 3 billion people. > 3. The name has to change because it's trademarked. By Disney. Who are Red Hat customers and partners, and good friends in the open source ecosystem. And Red Hat's logo is (for some reason still) on the bottom of this page. > 4. I can imagine how the `Hitchhiker's Guide` on the front page could be replaced with `Community Handbook`, and it would wrap around all the HtG content. But we need some tooling work to create an editorial workflow so we can scale documentation. This is because the docs are embedded in various repos as MD and I guess IPYNB files across a growing number of repos. That is the model, correct, having docs directly in each repo? That 5's fine but we need to centralize the editorial review and publish workflow if you want docs to scale beyond just you all. > > **Karsten (quaid) Wade wrote:** > Do we have any kind of actual learning path for SREs at this point? > > What do the Jupyter Notebooks in the hitchhiker's guide do? > > If we're keeping SRE doss as Jupyter Notebooks, can we separate out the content as MarkDown files for ease of contribution? > > We need a rudimentary self-learning pathway for SREs that links directly from the front page, until we can replace that with the learning paths from our curriculum designers. Right? Isn't that what the hitchhiker's guide work was about? > > So, I went to crosslink into the Community Handbook and found it seems to all be Jupyter Notebooks? Which is not a low-barrier to contributing method. For an all-contributors guide maintained by all contributors, we need a much lower barrier to contributing. I had expected to find a way to cross-link it to the MD files in /community-handbook/ so I could ask for help in getting operate-first.cloud/community-handbook/ running. But I don't see how to fold in high-barrier to contribution and use tooling like Jupyter Notebook. > > Then it's unclear to me what the "Hitchhiker's Guide" does as a link on the front page. It looks like all of the team procedures w