new badges for stats peer review

[mpadge]

@sckott We would now like to incorporate the new stats peer review badges here. There are 3 new badges:

Your current code here simply associates the standard "Approved" label with the standard "Peer-reviewed" svg. We have not to date considered different versions of the "Approved" label itself; only that approval will be triggered by different commands of @ropensci-review-bot approve bronze/silver/gold. I guess the easiest thing would be to translate these into distinct issue labels, so they could then be directly used in the update_badges.rb script? That could all then simply extend the current "status" variable, so it could take these three new values as extensions of "approved". The JSON format would then not have to change. Does that sound workable?

---

A second problem we would then have would be the versioning. As standards develop, numbers of badges will increase, retaining one set of 3 badges for each preceding iteration of standards. We're still working out the best way to track and record this. The version data should be automatically generated in response to the final approve command, so the bot will declare the current version of standards for which the software is approved. I guess we could do that in a standard way so it could be grepped in update_badges to extract the associated version? There'll also be possibilities for authors to update their software to align with more recent standards, so the command to approve standards alignment will then be repeated and will generate a second version of the same statement.

One direct way to achieve this would be to also have versioned forms of the issue labels themselves, so there would be approved-stats-bronze-0.0.1 and so on. Bot commands could be used to directly update the issue labels, and so update_badges would then only need to find an approved label and then map the full name of the label on to the name of associated svgs. Only problem there would be proliferation of issue labels, so would definitely like your feedback there before thinking any further. Ping @noamross

[noamross]

Good summary, Mark. I don't see a problem with a greater proliferation of badges. Go with that at first pass.

That said, it would be good to think about an immutable registry that associates an approval with a hash of both the package commit/version and the standards commit/version.

[sckott]

The first part sounds good mark.

A few things to note in case you weren't aware:

• svgs that are actually used are in the gh-pages branch https://github.com/ropensci-org/badges/tree/gh-pages
• this repo uses the CNAME badges.ropensci.org. is that okay or do you want a different base url?
• perhaps its easiest to have a separate ruby script for stats software review, copying update_badges.rb, then run it in the action file after or before this line https://github.com/ropensci-org/badges/blob/main/.github/workflows/badges.yml#L26 - might be a bit cleaner to separate concerns a bit while keeping - one caveat is that update_badges.rb currently cleans out the svgs folder before writing to it https://github.com/ropensci-org/badges/blob/main/update_badges.rb#L8 - easy to fix but just to be aware in case separate scripts are used
• this currently runs on a cron schedule https://github.com/ropensci-org/badges/blob/main/.github/workflows/badges.yml#L8 - do you need badges to be updated/changed in response to the bot? i imagine that can be done but is not set up to do that now

[maelle]

That said, it would be good to think about an immutable registry that associates an approval with a hash of both the package commit/version and the standards commit/version.

@noamross is this something that should be added as a feature for the Airtable-related buffy stuff?

[sckott]

@mpadge can work on this now. Any response to my questions above?

Some more thoughts/questions:

• Will need to address the fact that Github Actions are automatically disabled after I think 60 days. For example, it was disabled until just an hour or so ago for 18 days. i.e., badges hadn't been updated in 18 days. Perhaps a solution is a cron job on a server that can check if any important action workflows are disabled, and if so, enable them again - via the endpoint https://docs.github.com/en/rest/reference/actions#enable-a-workflow
• Before I proceed in drafting some code, has your thinking changed from your first comment above on from what information badges are created? I don't know that you have to have the same workflow for both regular and stats review. E.g., @ropensci-review-bot approve bronze/silver/gold could also trigger a build here to make/update a badge

[mpadge]

Thanks @sckott, some responses to your questions:

this repo uses the CNAME badges.ropensci.org. is that okay or do you want a different base url?

That one's great

perhaps its easiest to have a separate ruby script for stats software review, copying update_badges.rb, then run it in the action file after or before this line https://github.com/ropensci-org/badges/blob/main/.github/workflows/badges.yml#L26 - might be a bit cleaner to separate concerns a bit while keeping

I don't have a firm opinion either way here, although suspect it might actually be cleaner simply modifying current script, because it would then be more obvious how the two workflows are related. We'd then just need to extend this line:

https://github.com/ropensci-org/badges/blob/0047587bfa4d5ba8215c4589b01da7ae044db787/update_badges.rb#L18

to concatenate the new badges into iss_peer_rev. The stats_checks array that follows would then just need the new lines for grep(/approved-stats-bronze-0.0.1/) and so on. So modifying current script would require only two additional lines for each new badge type, whereas script duplications would be adding far more code. Does that sound about right?

this currently runs on a cron schedule https://github.com/ropensci-org/badges/blob/main/.github/workflows/badges.yml#L8 - do you need badges to be updated/changed in response to the bot? i imagine that can be done but is not set up to do that now

Daily updates should be fine.

---

As for workflow updates and re-enabling, i guess that would be an argument for at least having any @ropensci-review-bot approve commands enable the workflow, right? That should be easy for @xuanxu to incorporate

[sckott]

Okay, I'll start a new branch, modifying the current ruby script, and we can discuss in PR

Do any of the labels approved-stats-bronze etc. exist yet?

that would be an argument for at least having any @ropensci-review-bot approve commands enable the workflow

sounds like a good idea

[mpadge]

Do any of the labels approved-stats-bronze etc. exist yet?

Nope, not yet, because we need this discussion to work out whether the labels themselves need to be versioned, or whether we can get away with just having generic labels. The latter would be better, but we then somehow need a way for the package.json to be constructed with a record of current or last compliant version.

[sckott]

... with a record of current or last compliant version

How do I get that information?

[sckott]

@mpadge Okay, now I've edited code in the Ruby script to add a new field to the json file, so each entry looks like:

{
    "pkgname": "bizbaz",
    "submitter": "janedoe",
    "iss_no": 10,
    "status": "reviewed",
    "version": "0.1.14",
    "stats_version": "0.0.1"
}

Thoughts?

[mpadge]

That looks like it should all do the job. Sorry i somehow missed your question on standards version - the definitive value will be that on the badge, so this should all work perfectly as is. You only need to worry about the badge; and tweaks to approved version compliance will be handled by buffy, via commands that themselves will directly change the badge.

[sckott]

Okay, sounds good. Thanks.