Provide interactive plans for peer review

[brynwhyman]

Problem

A key part of the release process is peer review of the release plan. This requires the peer reviewer to work through the plan module-by-module, confirming that the plan accurately reflects whether a new version is needed, and what type of release it should be (minor vs. patch).

The UI we have for visualising the plan is fairly readable for the purpose of creating the plan, but leaves a lot to be desired when reviewing it:

• The easiest way to accurately get this visualisation into an external document/wiki is to screenshot it, which makes the plan completely non-interactive.
• Even copy/pasting the text content, or spinning up a copy of the plan, doesn't provide any useful URLs for doubling checking the diffs, making this a manual operation. There are some external, hand-rolled scripts floating around to try and reduce this toil, but they're very adhoc / buggy and ideally this should be a core capability of Cow (or at least a robust, supported tool).
• CWP releases are based on cwp-recipe-kitchen-sink, which results in giant plans that begin to exhaust the readability of the tree view. Ideally we would be able to collapse parts of the tree, or filter it.

Potential solutions

The functionality we want to expose to the reviewer is:

• The same tree view that is rendered in the CLI, without spinning up a release in Cow
• GitHub links to repos, version comparisons, releases, etc. for each dependency
• (Stretch-goal) Checklist functionality so the reviewer can easily keep track of their progress
• (Stretch-goal) Ability to collapse / filter the tree to isolate individual dependencies for review
• (Stretch-goal) Make it easy to share the plan

A few different approaches have been put forward:

• Option 1: Have Cow generate a static HTML / Markdown document alongside .cow.pat.json with a snapshot of the interactive tree
• Option 2: Create a standalone web app that reads the contents of a .cow.pat.json file and outputs the interactive tree
  • This could either be a centrally hosted app that handles file uploads, or a local-first app that reads the file directly from disk and can be statically published when required
• Option 3: A native app using a cross-platform framework like Qt

Actions

• [ ] Decide which Option to pursue
• [ ] Implement PoC to get feedback and trial in next release
• [ ] Ship final product with documentation in Cow repository / Silverstripe release guides

[brynwhyman]

https://github.com/silverstripe/cow/issues/134 would of course be a great start though!

[ScopeyNZ]

It would probably be possible to build a quick standalone tool that can read/display a cow.pat.json (the JSON representation of a release plan). I think that would be the quickest road to victory, so to speak. A quick Vue or React app could do it. Could even be run locally with an npm install @silverstripe/cow-plan-viewer or something.

You could also probably do some cool stuff with the GitHub GraphQL API.

[Cheddam]

Generating an interactive UI (static HTML or otherwise) that has links to the repos, directly to change comparisons, to existing tags, etc. is the core goal here - it should replace the screenshot of the plan from the CLI tool with something more useful to a reviewer to reduce the effort required from them. Definitely wouldn't be opposed to whipping up a local SPA to render this, but perhaps just outputting an HTML / Markdown document alongside .cow.pat.json is a sufficient starting point. (We already have Twig available in Cow for this purpose).

[robbieaverill]

Would be cool if you could permalink those views too, so you could refer back to them in historic release plans from Confluence. Screenshots do well there at the moment.

[ScopeyNZ]

The advantage of approaching this with the intent to make an SPA is that you're way more flexible on hosting options (if you want this to be on the internet).

I was thinking that you have something (either spun up locally or hosted statically) that has a textarea where you can paste the contents of a .cow.pat.json. This would then render all the relevant front-end elements.

You could then expand this later to have some historical repository of cow pats, edit the JSON file and download it back from the SPA using some new-age JS APIs, or maybe whip up a quick node.js endpoint for live-edits if you want to be able to edit from a UI locally.

It just gives more options.

[tractorcow]

You could build a nice frontend on https://www.qt.io/ (using any language you want really).

[Cheddam]

I've given the original post an overhaul describing the fundamental goals it aims to tackle, and outlining the implementation options that have been discussed so far.

My take on this is that Option 1 would probably be the least time-intensive to implement, while Option 2 will provide the greatest potential benefit. Option 3 could be an interesting learning opportunity, but would likely result in excessive build time and would be hardest to maintain.

I'd like to kick off work on a PoC for Option 2. This won't happen immediately, so additional feedback is welcome in the meantime.

[chillu]

Another angle on this: It'll be an audit artefact in terms of change control which we need to retain. Git tags are a good start there, but ideally it's a list of commits in a version controlled document (e.g. Confluence) where we can document explicit sign off against this particular set of commits.

I'm not sure how much that overlaps with this issue, might be separate - but I see two goals here: Effective review/execution of a change, and durable/auditable documentation of that change.

I'm suspect of creating a web application which is used by around five people over time, for the sole purpose of some internal release tooling. A HTML doc with checkboxes which you can copy/paste into Confluence seems to get pretty close to that, at much lower investment? Keep in mind that there'll be more controls over time types of systems we can involve in creating audit artefacts like a recipe release in the future, so don't assume that we can just chuck a tool like that on zeit.co. In this context, simple = good.

[ScopeyNZ]

document explicit sign off against this particular set of commits

The tricky part about this is that Cow doesn't (by design mostly) tie down to specific commits, and doesn't currently generate an artefact of what commits it will actually tag until you do it. That would have to be a separate task if required.

I know the Creative Commoners did have a peer review process involving getting someone to review the cow pat, that was tracked in Confluence, but was fielded to the team in Slack.

[chillu]

OK thanks for clarifying, sounds like a separate task them. Been thinking a lot about more documented change management as part of an upcoming audit here, so I'll raise this closer to when it's required again. Sounds like this issue here can move independently, which is good :)

[Cheddam]

If we go the route of a hosted tool, we can make plans uploaded to it immutable, so that if changes are required through the peer review process, a new plan with a new URL is produced, and this can be versioned by way of updating the Confluence document. Alternatively, we can continue to upload the raw plan to the Confluence document and have the interactive tool be stateless (loading plans into memory and displaying them, rather than storing them itself).

I concede that this might be overkill for an internal tool, but the more confidence we can have that our plans are valid and that we are peer reviewing them correctly, and the more toil we can cut out of the review process, the better our releases will be - and this was raised as part of a focus on 'better releases'.

I plan to tackle the core piece - turning a .cow.pat.json into an interactive tree - on Friday. Based on how far I get, we can make a call on whether an external tool is useful / viable, and if not we can fall back to Option 1.

[Cheddam]

Unfortunately I didn't get a lot of time to dive into this today, but here's my progress so far:

• I've produced a super barebones Vue app that takes a .cow.pat.json file as input and renders it into a tree view (using CSS Subgrid to get the detail columns aligning correctly, sorry to every browser but Firefox for now 😅)
• Unfortunately, the data contained in the plan schema is currently too barebones to produce the intended comparison links, or to even render prior versions for some dependencies, so we'll need to expand the cowpat schema and get Cow to consistently populate version information.
• I've also been digging into the Atlassian Connect APIs, which would allow us to expose this view as a custom content type for Confluence, rather than a separate web app. The samples are very simple Node apps that render the content on-demand, so this wouldn't be too tough to develop / maintain but I'd need to find a hosting solution.