Split Qubes docs into "core" and "community" docs

[ghost]

This will likely be a controversial topic, but let's float the idea.

tl;dr; imho it would make sense to split the docs into "core", well-reviewed, security-sensitive docs, and "community" reviewed docs that are more "peripheral" to Qubes. This should help:

• increase the number of first-time contributors - who could then become regular contributors to the project
• help contributors with doc guidelines and improve the quality of the docs overall; the "community" could be used as a staging area similar to the staging branch of the linux kernel
• lower @marmarek 's workload
• increase (and make easily found/searchable) the number of docs/resources related to Qubes OS's usage.

[EDIT] An alternative would be to leave the docs as-is, and put links to "community" resources somewhere in the official docs.

---

Issues with the current docs:

• the PR submission process isn't easy for git noobs. At least it wasn't for me - I had to learn git only to be able to improve the docs, it did require time and motivation, hopefully that was at a time when I could spend more spare time on Qubes than I do now. That's likely not the case for most users, some of them who could have provided helpful improvements but are/were put off by the seemingly convoluted process. (Sure, once you know git it is easy but then not everyone knows git nor is interested in learning it).
• @marmarek is more or less the only technical reviewer of doc PRs nowadays and his workload makes him a bottleneck; PRs can stay for weeks/months before being accepted. That can be a bit discouraging for users who have put some time in trying to improve the project. (to be clear I'm not complaining - just stating what I see).
• some official docs have nothing to do with Qubes and clutter the doc ToC.

The Qubes Community project tries to solve (some of) the issues above. The project is very slowly getting some traction but judging by questions asked on the ML and issues which are already answered in details in docs in the project, most people aren't aware of this effort.

There are also great third party resources (eg @tasket 's contributions, @unman 's repositories, ...) that users aren't aware of because they're not advertised anywhere in the official site - and rightfully so, because that could give a false sense of endorsement and/or security. Having links to those resources in a "community" resources page, with the usual disclaimer, would definitely increase the visibility of such efforts.

Given all of the above, wouldn't it make sense to have only a small amount of "core" official docs that are carefully reviewed by the Qubes team and thus have a higher "security trust level", and have a Qubes endorsed community doc project (not necessarily the one mentioned above), administered by proficient/long-time qubes users - maybe picked by the Qubes team - where the submission rules/quality would be relaxed, and where it would be clear for users that those community docs have a lower "trust level" ?

As noted above, this would:

• allow people not familiar to git to contribute stuff, and maybe learn git in the process until they're confident enough to send PRs (both the the community and official docs).
• allow people to contribute lower quality/incomplete/... but helpful docs (eg. specifics of installing [whateverOS] as a qube VM).
• streamline the official docs structure (and thus making it easier to parse) by keeping only the documentation specific to Qubes OS, on which the community docs are based. Some of the existing docs could me moved - eg. configuration of OSes in (H)VMs, XFCE/KDE features that aren't specific to Qubes, DPI in VMs, setting a dark background, etcetera.

Thoughts ?

[andrewdavidwong]

I somewhat like the idea of limiting the scope of official security-reviewed (in practice, @marmarek reviewed) documentation to just core functionality of Qubes OS. For example, docs about the Fedora and Debian TemplateVMs, DisposableVMs, USB VMs, Split GPG, and AEM are clearly core, while docs about i3, Kali, YubiKey, and ArchLinux are clearly not. That's not to say the latter are not valuable or important. It's just that the former are components of the OS developed by the Qubes team, whereas the latter are not. The latter are important and valuable things that many (but not all) Qubes users want to be able to do with the system that originate from outside of the Qubes OS Project.

[andrewdavidwong]

Related to: #3629

[andrewdavidwong]

I've created a new section for external documentation and reorganized the documentation accordingly:

https://www.qubes-os.org/doc/#external-documentation

For now, I'm just displaying a warning on all of the "external" doc pages, even if they're technically still internal to qubes-doc. This way, we can be a bit more liberal about accepting PRs against those documents, and we have the option of gradually migrating them over to the Qubes-Community docs.

[andrewdavidwong]

Now that we have the External Documentation section, I'm thinking about how best to deal with the backlog of PRs against qubes-doc and how best to expose the contents of https://github.com/Qubes-Community/Contents/ to visitors of https://www.qubes-os.org/doc/.

One option is to have a link on https://www.qubes-os.org/doc/ for each page in https://github.com/Qubes-Community/Contents/, which would essentially be copying the contents of https://github.com/Qubes-Community/Contents/blob/master/docs/README.md into https://github.com/QubesOS/qubes-doc/blob/master/doc.md. Then, when new pages are added to https://github.com/Qubes-Community/Contents/, folks would just submit PRs to add lines to https://github.com/QubesOS/qubes-doc/blob/master/doc.md. If we were to do this, it would make sense to somehow move all of the open qubes-doc PRs about "external" content to https://github.com/Qubes-Community/Contents/.

@taradiddles, @awokd, @one7two99: What are your thoughts on this?

[awokd]

Nice reorganization, @andrewdavidwong. I'm a bit surprised changing time zones aren't somewhere in the official documents! Would the end goal be that all those External documents are migrated to Qubes-Community? Would kind of have to, to handle PRs about them there.

How we handle linking to the offsite documents could depend on how much distance the QubesOS project wants to put between itself and them. For example, with just a single link to Qubes-Community and no specific documents, it's quite clear they are external. This would also be less maintenance overhead (1 PR vs. 2), but drawback is people might not see the document they want and not click through. Is there a safe, dynamic way to generate that list so only one master list needs to be maintained under Qubes-Community?

I'm fine with starting with your proposal. Adding and deleting entire documents is relatively rare, so 2 PRs isn't a great stretch.

[andrewdavidwong]

Nice reorganization, @andrewdavidwong.

Thanks!

I'm a bit surprised changing time zones aren't somewhere in the official documents!

Well, Changing your Time Zone just covers the basics of using the timedatectl command, which is not at all Qubes-specific. I don't see anything on that page that's Qubes-specific, let alone that deals with the core functionality of Qubes. Am I missing something?

As far as I know, for the average Qubes user, there's nothing special about changing your time zone in Qubes. You just change the timezone in the dom0 the way you would in regular Fedora, and you're done. At least, this has been my experience.

Would the end goal be that all those External documents are migrated to Qubes-Community? Would kind of have to, to handle PRs about them there.

That's my initial proposal, pending feedback from you all.

How we handle linking to the offsite documents could depend on how much distance the QubesOS project wants to put between itself and them. For example, with just a single link to Qubes-Community and no specific documents, it's quite clear they are external. This would also be less maintenance overhead (1 PR vs. 2), but drawback is people might not see the document they want and not click through.

Right. There's also the problem that documents on other websites (e.g., this one) would be more discoverable than the documents on Qubes-Community, which seems undesirable to me.

I suppose another option is to move those to the Qubes-Community list, as well. Then Qubes-Community would become the place for all external docs, and we'd just have one link there. This would reduce the maintenance overhead of having to make PRs against doc.md.

Is there a safe, dynamic way to generate that list so only one master list needs to be maintained under Qubes-Community?

Not that I know of.

I'm fine with starting with your proposal. Adding and deleting entire documents is relatively rare, so 2 PRs isn't a great stretch.

What are your thoughts on just linking once to Qubes-Community and keeping the master list of external docs there?

CC: @marmarek

[awokd]

I remember when first starting with R3.2, changing the timezone wasn't intuitive. In Windows it's accomplished by right-clicking the clock and going to Properties. Doing the same in Qubes doesn't work well. Sorry, am getting this issue off track.

A single link through to everything works for me as well; I'm good either way.

[andrewdavidwong]

This issue has been open for feedback for quite a long time now with no major objections, so I think I'm going to finally move the External documents over to Qubes Community Collaboration. It should be pretty straightforward. Here's what I'm thinking.

For each page in External Docs:

1. Convert relative links to absolute links (https://github.com/QubesOS/qubes-issues/issues/4693#issuecomment-698132058).
2. Move the .md file to an appropriate subdirectory of https://github.com/Qubes-Community/Contents/tree/master/docs [edit: in a way that preserves its history (see below)].
3. Replace the contents of the .md file in qubes-doc with a redirect_to the new URL in https://github.com/Qubes-Community/Contents/tree/master/docs.
4. Update any internal links to that page to the new URL.

Every time someone wants to add/remove an external doc, simply submit a PR against https://github.com/QubesOS/qubes-doc/blob/master/doc.md to add/remove a link to the doc in https://github.com/Qubes-Community/Contents/tree/master/docs.

@taradiddles, @awokd, any final thoughts?

[awokd]

I'm fine with this, and can help move some/all of these if you like. If some, how would you like to divvy them up? Other option might be the "how to" section in the Discourse group, but from the lack of responses to https://www.mail-archive.com/qubes-devel@googlegroups.com/msg04517.html, I took it as a "no".

[unman]

On Tue, Sep 22, 2020 at 09:28:22AM -0700, Andrew David Wong wrote:

This issue has been open for feedback for quite a long time now with no major objections, so I think I'm going to finally move the External documents over to Qubes Community Collaboration. It should be pretty straightforward. Here's what I'm thinking.

For each page in External Docs:

1. Copy the .md file to an appropriate subdirectory of https://github.com/Qubes-Community/Contents/tree/master/docs and commit.
2. Replace the contents of the .md file in qubes-doc with a redirect_to the new URL in https://github.com/Qubes-Community/Contents/tree/master/docs.
3. Update any internal links to that page to the new URL.

Every time someone wants to add/remove an external doc, simply submit a PR against https://github.com/QubesOS/qubes-doc/blob/master/doc.md to add/remove a link to the doc in https://github.com/Qubes-Community/Contents/tree/master/docs.

@taradiddles, @awokd, any final thoughts?

You'll lose history wont you?

[andrewdavidwong]

You'll lose history wont you?

No, due to the nature of Git, no history will be lost. However, after the transition, if you wanted to see the pre-transition history, you'd have to look in qubes-doc rather than https://github.com/Qubes-Community/Contents.

[andrewdavidwong]

Actually, we should be able to move the files in a way that also transfers their history:

https://stackoverflow.com/questions/1365541/how-to-move-files-from-one-git-repo-to-another-not-a-clone-preserving-history https://medium.com/@ayushya/move-directory-from-one-repository-to-another-preserving-git-history-d210fa049d4b

[deeplow]

So does this mean that the content will then be available at https://qubes-community.github.io/? Or will people be looking at https://github.com/Qubes-Community/Contents directly?

If the former is the case, then it'll still need a bit of work to make an index and content reachable in general.

Idea

Here's an idea: what about making https://qubes-community.github.io/ look like the documentation section of the website (along with all the styling) but giving it a different color to make it distinct?

Here's a picture of what it could look like:

edit: forgot to add in the picture "Forum / Mailing Lists"

A few advantages:

• website is already made and looks pretty nice
• integration with github already exists (edit this page link)
• users are already familar with navigating on the official docs so this woudn't be any different

Any thoughts? @andrewdavidwong

[andrewdavidwong]

That looks very nice! We might need to visually distinguish it even more to avoid users being confused about whether they're looking at the official site.

[deeplow]

That looks very nice!

Thanks. In a significant way thanks to you :) All I did was changing the colors :P

We might need to visually distinguish it even more to avoid users being confused about whether they're looking at the official site.

Totally. Perhaps something else that could be done would be adding a link "official website" in the top right corner like so @andrewdavidwong

[awokd]

@deeplow, would you be able to make the appropriate commits to the qubes-community github.io site? Someone smarter than me set up the web page view originally, but has since left for greener pastures. Could use another admin too, actually, to help approve commits if you're willing. It's pretty light volume so far.

@andrewdavidwong, I'll attempt to follow the steps in that Medium article on my own Github site where I'm not as concerned about maintaining history. If it works, we can coordinate further on go-live/cutover steps to qubes-community.

[deeplow]

would you be able to make the appropriate commits to the qubes-community github.io site?

Hi @awokd. I'd be interested implementing the above suggestion if we choose to go in that direction (after addressing all risks of users think that's the official website). So it's fine by me to do that initial push, especially as the website is already fully constructed and documented.

Could use another admin too, actually, to help approve commits if you're willing. It's pretty light volume so far.

Thanks for the invitation. The forum has been keeping me pretty busy (but I like it). So unfortunately I cannot commit to helping approve PRs on a regular basis at this time. But I can however make an effort to help from time to time.

Are currently the only admin doing the reviewing and website maintenance?

But I'll also be helping indirectly. Some guides start in the "How-Tos" section on the forum and there I'll help with the formatting and take a look at them. I will later on steer the users who produced How-Tos into opening a PR and going through the review process. Since some discussion should have already happened on the forum and the formatting should be mostly handled (markdown as well) should ease the PR review process a bit.

[deeplow]

I've just remembered that there probably will be the need to replace all relative links that point from the external documentation to the official one @andrewdavidwong

[deeplow]

@awokd @andrewdavidwong won't it be better to split the documentation from the Contents?

I would suggest there is one repo only for the documentation qubes-community-doc and then one qubes-community-utils with small scripts. Any project larger than a simple script could have its own repo.

[andrewdavidwong]

I've just remembered that there probably will be the need to replace all relative links that point from the external documentation to the official one

Yes, good point.

won't it be better to split the documentation from the Contents? [...]

I leave such decisions to @awokd as (current sole) admin. I would like to respect the autonomy of that organization as an independent entity.

[andrewdavidwong]

Reproducing my comment from https://github.com/Qubes-Community/Qubes-Community.github.io/issues/2#issuecomment-699498161:

After considering this a bit more, I think that making a whole new website might be overkill. Even if the initial setup is made "easy" by just copying and tweaking the Qubes website, there will still be ongoing maintenance required, which might be hard since it's all volunteer-based. There could easily come a point in the future when something breaks and no one has time to fix it, so it remains broken for an extended period of time while the doc pages are unreadable by the general public.

Instead, why not take a low-maintenance approach by simply using what GitHub already gives us? Here's a random example doc from https://github.com/Qubes-Community/Contents/:

https://github.com/Qubes-Community/Contents/blob/master/docs/configuration/zoom_dispvm.md

When you view this page, the Markdown is rendered very similarly to the way it would appear on the Qubes website (or on the new website we're considering). It's as readable as any normal website, and it's clear that this is not the official Qubes website, so there's no risk of confusion. Moreover, it requires zero maintenance, since rendering the Markdown this way is built into GitHub. Only the underlying Markdown files and Git repo have to be maintained as usual.

Moreover, this makes it very transparent when we're linking from qubes-doc/doc.md to docs in https://github.com/Qubes-Community/Contents/. We just link to the master branch (as in the link above), and it automatically reflects the latest version as the underlying file gets updated. Since we're not messing around with custom permalinks or redirects, there are fewer things that can go wrong, fewer dead links, less overall hassle and ongoing maintenance burden.

[andrewdavidwong]

In order to convert the relative links into absolute links, I believe it should be sufficient to make the following replacements:

• ](/ to ](https://www.qubes-os.org/
• ]: / to ]: https://www.qubes-os.org/

But let me know if anyone has a better idea.

[andrewdavidwong]

Another question concerns attachments, such as images. Currently, they're all stored in qubes-attachment. It would probably make more sense for the attachments that belong to each doc to be stored in that org's repo.

[awokd]

@awokd @andrewdavidwong won't it be better to split the documentation from the Contents?

I would suggest there is one repo only for the documentation qubes-community-doc and then one qubes-community-utils with small scripts. Any project larger than a simple script could have its own repo.

We had originally attempted something along those lines for proposed edits to the documents, but it was overkill for what we needed at the time. Might make sense here, though. Let's continue discussing over on https://github.com/Qubes-Community/Qubes-Community.github.io/issues/2?

[andrewdavidwong]

I was able to preserve partial history in the target repo. Unfortunately, due to a limitation in git, history cannot be preserved back through previous file/directory renames. (For details, see all the answers and comments on this StackOverflow question.) However, as mentioned before, no history is actually lost. At worst, it's just split across the two repos.

[andrewdavidwong]

The migration is now complete. If anyone sees any problems or anything else that should be done, please leave a comment.

[deeplow]

@andrewdavidwong What about the index? It's still available on https://www.qubes-os.org/doc/#external-documentation but things will be probably changing. Do you accept PRs to change the links to the community docs?

For example. Some of us have been collaborating on a split-ssh guide which should replace kushada's blog post linked on the qubes doc

[andrewdavidwong]

@andrewdavidwong What about the index? It's still available on https://www.qubes-os.org/doc/#external-documentation but things will be probably changing. Do you accept PRs to change the links to the community docs?

For example. Some of us have been collaborating on a split-ssh guide which should replace kushada's blog post linked on the qubes doc

Yes. The idea is that the maintainers or editors in chief of Qubes-Community will decide which community docs are ready to show off to the wider world, then submit PRs to qubes-doc to link directly to those from the index. Likewise, they should also remove/replace old or bad Qubes-Community links, as appropriate.

I'll also add add this to the doc guidelines.

CC: @awokd

[awokd]

The idea is that the maintainers or editors in chief of Qubes-Community will decide which community docs are ready to show off to the wider world, then submit PRs to qubes-doc to link directly to those from the index. Likewise, they should also remove/replace old or bad Qubes-Community links, as appropriate.

Thanks, missed that part.