Write an ADR around Docs Hub being a separate repo

[data-sync-user]

I only noticed this today, but I noticed we decided to go ahead with the docs hub being in a separate repo. I am pretty against this in general because it has impacts beyond just deployment that get introduced (like Jira sync, permissions management, PR templates etc.)

Happy to hear arguments/concerns here (let's discuss next week)

┆Issue is synchronized with this Jira Task ┆Issue Number: EXP-1042

[data-sync-user]

➤ Kate Hudson commented:

cc [~lzugai@mozilla.com] [~lorchard@mozilla.com] [~jheavener@mozilla.com] [~jkerim@mozilla.com]

[jodyheavener]

@k88hudson here's a pretty long thread for context as to how we ended up with a new repo: https://mozilla.slack.com/archives/CANCHH6R1/p1614624609195500

[jodyheavener]

Just so we're not losing everything in the depths of Slack thread hell, here's my attempt at distilling some of the points from the linked thread.

Thoughts on using a separate repo:

• It's entirely separate from the main repo, allowing us to set up Circle or GH Actions as we see fit
• It also means inviting people to the repo, security and access concerns, potential Jira setup
• Code and setup live on main branch, static site and docs get deployed and made available on the gh-pages branch. gh-pages never has to be touched.
• You couldn't land changes that introduce code AND docs in one PR

Thoughts on using the experimenter repo without a dedicated docs branch:

(Use the experimenter repo, but put the Docusaurus code and docs under its own directory like app/docs)

• It will be tricky to set up CI in such a way that it only runs our workflows when code changes land and not when doc-related changes land. CircleCI cannot filter based on path changes, so we'd need to switch all our current workflows to use a custom script that compares the files changed in the diff and then calls jobs using the CircleCI API if the files don't touch doc paths.
• Allows us to house everything, code and Dependabot, under one repo
• You could land changes that introduce code AND docs in one PR
• Similar to the separate repo, doc changes can be deployed to and served from a dedicated gh-pages branch.
• In order to use yarn workspaces the code would need to live under the app/ directory, and not at the root

Thoughts on using the experimenter repo WITH a dedicated docs branch:

(Use the experimenter repo, but dedicate a branch entirely to Docusarus code, where the branch and main never interact)

• Unsure if CircleCI can handle two distinct configs (one for main, one for gh-pages) or a config for main and no config for gh-pages (if we used GH Actions for that branch).
• It's basically a separate repo hiding inside our main repo (this has good and bad implications)
• Auto-deploying built code to the same branch you're working on (gh-pages) will add commits to the same branch and potentially make things messy
• You couldn't land changes that introduce code AND docs in one PR
• You've got all these PRs mixed together, that are talking about completely distinct source paths and ci tasks that are unrelated

[data-sync-user]

➤ Kate Hudson commented:

Ok, so my main concern about adding new repos (to Nimbus, but I have to admit, in general) is that while it seems much easier/simpler/cleaner up front, it adds unexpected cultural and technical overhead in the long term – for example, things like setting up the Jira integration, maintaining the circle integration, github permissions, etc. I know docs are a little bit different v.s. actual code (for example, cross-dependency management and running the application end-to-end do not apply) but the overhead pieces stand.

All the reasons you've outlined for maintaining a separate branch make it seem like not an ideal solution. But where does the concern about running custom CI when code v.s. docs changes land come from? What would be the threshold of the volume of docs changes are where we would run into cost concerns, and are we likely to hit them?

[data-sync-user]

➤ Kate Hudson commented:

Re: Things needing to live under the app/ directory, is there a reason you couldn't just have it be a separate directory outside the workspace? Like, is that particularly bad/gross compared to it being in a separate repo? Or are there additional advantages (or bad things) about it being part of the workspace?

[data-sync-user]

➤ Tim Smith commented:

We can do a few things to reduce overhead:

• Disable issues on the new repo and track docs issues in the Experimenter repo instead. No changes would be needed to the Jira integration. Syntax like closes mozilla/fake-repo#1 in PRs works across repos.
• Use the project-nimbus team to manage permissions, as we're already doing for the repos we have

I think having a docs-only branch will be confusing for contributors -- opening a pull request to a non-default branch is certainly possible but requires ticking boxes carefully.

Making a docs PR to the Experimenter repo will require contributors who are not members of the core team to comprehend a lot more context. It's actually pretty hard to find e.g. the ADRs in the Experimenter repo today. We could make that easier but there's still going to be a lot of stuff to ignore and if you're not an Experimenter engineer I think that's a lot of overhead.

[jodyheavener]

While I'm still leaning towards a separate repo at the moment I'm sort of flip-floppy on this. I think I am just dreading trying to break down Circle stuff if we decide to do the same repo. Some responses to your comments...

it adds unexpected cultural and technical overhead in the long term – for example, things like setting up the Jira integration, maintaining the circle integration, github permissions, etc.

Fair point, we do have to keep an eye out for Dependabot PRs now. FWIW, though, there's no Circle integration here, it's a fairly simple GitHub Action, and GitHub permissions are as simple as adding @mozilla/project-nimbus to the repo now. Jira would def be some overhead, but I question if we even need integration for it here (Edit: I like @tdsmith's idea of disabling repo issues and restricting them to the experimenter repo).

is there a reason you couldn't just have it be a separate directory outside the workspace? Like, is that particularly bad/gross compared to it being in a separate repo?

Yarn Workspaces require all Node apps to be nested within the folder that the main package.json file lives, otherwise you will end up with two spots where you need to yarn install (the first time, but also for any changed dependencies), as well as two distinct node_modules folders. This isn't the end of the world but could lead to confusion. With a separate repo it's clear there will be a separate install process.

But where does the concern about running custom CI when code v.s. docs changes land come from? What would be the threshold of the volume of docs changes are where we would run into cost concerns, and are we likely to hit them?

I don't know what the threshold would be, Jared would probably be better to answer that, but it generally feels gross to me that we'd run all/most workflows when you push and merge into main (so at minimum twice) when they didn't need to be run at all. It also adds a significantly longer wait time before you can merge.

[LZoog]

@jodyheavener because you've already done 95% of the work, this could be a good, (quick-ish) ADR to get in!

CircleCI cannot filter based on path changes, so we'd need to switch all our current workflows to use a custom script that compares the files changed in the diff and then calls jobs using the CircleCI API if the files don't touch doc paths.

Is this actually impossible to have it filter at the CI level, or is it something we could somehow build with not a super high level of effort? It very well may be, just want to make sure. I know you've looked into this further than I have.

I do like that, with a new repo, subscribing for notifications is easier, and we won't have two dependabots in Experimenter. In my original ADR, I suggested we have a separate branch in the Experimenter repo, though I think the biggest concern around having a separate doc-hub branch was that it may not be conventional to have a totally different branch maintained separately from main and the more we discussed it, the more it started to have that bad code smell (as Jody pointed out with, "it's basically a separate repo hiding inside our main repo") and CI challenges.

I wonder if it's possible to add a new tab to the Experimenter repo titled "Doc Hub" that we could link to, or if that would be helpful at all for visibility. We can (and should!) definitely update the front-page README of Experimenter with a blurb about the docs being in another repo. There may be other ways we can mitigate the cultural and technical overhead as brought up by @k88hudson, like the comment that @tdsmith added, and we'll have to decide if we like filing issues in Experimenter for issues here, but I definitely see the downsides for having it in a separate repo.

With all of this said, I am still a little torn on the matter. It seems like the decision weighs on how much cultural and technical overhead we add by leaving it in a new repository vs the weirdness around having a branch unrelated to main and setting up CI/dependabot/notifications for users.

[data-sync-user]

➤ Les Orchard commented:

I guess my main opinion here is that these docs shouldn't live in the mozilla/experimenter repo - if only because they're not solely devoted to Experimenter Console. We're talking about having stuff in there from jetstream, experimenter, not sure what else will be in there. We've already got lots of other separate-but-related repos involved in the ecosystem, seems weird to me that these docs would live particularly in mozilla/experimenter

[data-sync-user]

➤ Kate Hudson commented:

Alright, decision here is since we feel like this will be not more expensive to reverse later,

• We will leave the separate repo
• Close down issues in it
• [~jheavener@mozilla.com]  is gonna write an ADR
• We're going to try it for a month, and see if the assertions hold true

Thanks folks!

[data-sync-user]

➤ Kate Hudson commented:

Alright, decision here is since we feel like this will be not more expensive to reverse later,

• We will leave the separate repo
• Close down issues in it
• [~jheavener@mozilla.com]  is gonna write an ADR
• We're going to try it for a month, and see if the assertions hold true

Thanks folks!