Versioning for stats badges

[mpadge]

@sckott @noamross Starting this in a separate issue to #2 to help contain this discussion in one place. The following is a suggestion for a viable workflow to capture versioning directly from the actual badges, which is a general approach that @noamross and i both agree is likely to be both more direct, and most clearly indicative of versioning.

1. The bot replies to editor command @ropensci-review-bot approve silver (for example) with approved silver v0.0.x, and labels the issue with the appropriately versioned badge which would all be of the form approved-bronze-v0.0.x
2. This requires the bot to somehow grab the current latest standard versions (@xuanxu can work out the best approach for that), in order to assign the appropriately versioned label.
3. update_badges.rb is triggered by that same bot response (as well as cron-ed on daily basis), and dumps the correspondingly graded and versioned svgs into the gh-pages branch.

The third step will require these modifications to update_badges.rb:

1. The script will have to match both colors as in current #3, as well as versions.
2. That svg_map will slowly have to be expanded to map on to all versioned forms of all badges.

The singular advantage of handling everything here through direct labelling of badges is that it enables the management of versioning to be handled entirely independent of this repo and allows update_badges.rb to be easily expanded to new versions though inserting additional single lines only in svg_map. (And FYI version alignment will always be handled in original review issues, including via yet-to-be-implemented bot commands like @ropensci-review-bot upgrade to gold @ropensci-review-bot downgrade to bronze. Those commands will trigger changes to the badges in the original issue, so those badges will always record current alignment in an ongoing manner.)

@sckott This is intended only as a suggested workflow, so please suggest any other/better approaches you might think of, or clarify anything i may have misunderstood here. Thanks!

[noamross]

This largely makes sense to me. Two questions, @mpadge:

• Does the editor include a version number? e.g., do they write @ropensci-review-bot approve silver v0.0.1, or just @ropensci-review-bot approve silver. I think it's the latter and that's your plan, which does reduce the cognitive overhead on the editors.
• Can the standard versions be reported, and thus extracted from the pkgcheck results in the thread? Because the standard versions should be coded into the package itself, via srr. Or should the standard version be extracted from the package, rather than the pkgcheck results in the thread?

[mpadge]

Does the editor include a version number? e.g., do they write @ropensci-review-bot approve silver v0.0.1, or just @ropensci-review-bot approve silver

Yes, definitely the latter. Editors should never have to worry about versions!

Can the standard versions be reported, and thus extracted from the pkgcheck results in the thread? Because the standard versions should be coded into the package itself, via srr. Or should the standard version be extracted from the package, rather than the pkgcheck results in the thread?

Standard versions are currently not reported, but that's a good idea, to be addressed via pkgcheck#22. Nevertheless, the only ultimate definition of standards compliance will remain the badge on the original review issue, and that can only be modified by editor commands because they are the only ones who can issue approve commands. The"official" way to extract current compliant standards versions for any package will therefore only ever be to query the badge on the original review issue. This means that the definitive versions of standards are not directly coded into packages, rather are recorded "externally" in a way that only editors can directly control. This ensures that assessments of compliance always require an approve command, which i think seems like the right way to manage this.

[sckott]

Thanks @mpadge

update_badges.rb is triggered by that same bot response

I imagine we can use this approach https://docs.github.com/en/rest/reference/repos#create-a-repository-dispatch-event of triggering a gh actions workflow from the bot

The third step will require these modifications to update_badges.rb

I can start working on those

svg_map will slowly have to be expanded to map on to all versioned forms of all badges

That means map values of e.g., svgs/approved-bronze-v0.0.1.svg, svgs/approved-gold-v0.3.0.svg. Correct?

---

It'd be neat if there was a repo with at least some of the labels (e.g, approved-bronze-v0.0.x) so I can get to working code faster. Maybe I'll make a dummy repo with labels and test on that

[mpadge]

That means map values of e.g., svgs/approved-bronze-v0.0.1.svg, svgs/approved-gold-v0.3.0.svg. Correct?

Yep, I've set up labels in ropenscilab/statistical-software-review. Let me know if there's anything else i can do to help. Thanks a load for the input!

[sckott]

Got it, thanks

[sckott]

@mpadge Okay, in recent commit now using test issues in https://github.com/ropenscilabs/statistical-software-review/ (issues 10, 11, and 12) with the approved gold/silver/bronze labels.

Looks like it's working well now

svg files for stats review pkgs look like "10_status_stats.svg" with the "_stats" part the only difference from svg file names for non-stats pkgs

If you have time/interest, you should be able to run ruby update_badges.rb

[mpadge]

Cool - update_badges worked perfectly; thanks!

[sckott]

Great!