Split the output to two reports, public and private

[QuintinWillison]

We intend to action this before we work on #5 (making this repository public).

Currently we have a single downstream repository which contains results for public and private repositories. It's called:

ably/repository-audit-report

We will rename that repository to something like:

ably/repository-audit-report-internal

Then create a second downstream repository under the old name which we will make public under #5.

Then this codebase is modified to push results for:

• public repositories to ably/repository-audit-report
• private repositories to ably/repository-audit-report-internal

The advantages of publishing the report for our public repositories in the public domain are:

• Transparency: show people that we care about this stuff and that we're happy to be held accountable and respond accordingly
• Illustration: demonstrate real-world output from this tool for others who may be considering using it to survey their own repositories

[paddybyers]

I did wonder if there was a way (and if it would be appropriate) for the results of the audit to go into the subject repo in some form. However, I do see that that wouldn't achieve the goal of giving an overview in one place. But it might be an approach whereby the results for public repos are made public (ie by having them attached to the repo itself somehow), avoiding the need for two summary reports.

Anyway, I support the goal of making the results public for the public repos, one way or another.

[QuintinWillison]

Thanks, @paddybyers ... I had similar thoughts and, broadly speaking, I think the logical way to achieve that could be by developing that out as a new feature of this tool, sitting alongside the existing functionality - i.e.:

1. This tool generates an additional report artifact, containing the check results matrix, that's designed to be machine-readable (i.e. probably json)
2. That machine-readable report is transformed to push somewhere that maintains state of checks for each repository - perhaps an S3 bucket with object prefixes including repository name and check name, serving an image object - I would ideally do this work under a parallel workflow as it may take a while to run, depending on how it's designed (i.e. it's something that could and should be allowed to fail independent of the core report generation)
3. For each target repository we add a status badge consuming those image objects (e.g. akin to this) - i.e.:
   • image URL something like: https://github-repository-audit-status.ably.com/badges/ably-js/checks/a.svg
   • clicking the link taking you, via absolute deep-link, to the correct heading in ably/repository-audit-report/ably.md

Then we get a touch point per-repo, only increasing visibility of the over-arching report and upstream tooling.

[QuintinWillison]

Actually, thinking further on my earlier comment on this issue, it wouldn't scale very well for there to be a badge for each individual check. There should be a single badge for each repository. It could reflect a pass percentage or other form of more nuanced score, aggregating all checks.

While my initial thoughts around canonical URLs for these images could stand - for example:

https://github-repository-audit-status.ably.com/badges/ably-js.svg

It strikes me that given we already have sdk.ably.com (via ably/sdk-upload-action), it could well be acceptable to have status badges served from there - for example:

https://sdk.ably.com/builds/ably/repository-audit/main/badges/ably-js.svg

Bearing in mind, however, that once we start using these across our repositories then that base URL will be baked into those repository codebases... meaning that if we decide we want to move where these are hosted at some point then we'll have work required to do that.

A solution to that could be another S3 bucket with website hosting enabled, presenting at github-repository-audit-status.ably.com or similar, using a conditional redirect to return 301 redirecting status badge request to source from sdk.ably.com. Though that API looks pretty limited (use some of the object prefix and name components as arguments to formulate the redirect URL?) and by the time we've started trying to engineer such indirections we may as well just push to that bucket anyway. 🤷