A separate paper on badges

[CaptainSifff]

@jcohen02 and @fer-rum want to write a separate paper on badges and gamification for software....

[CaptainSifff]

@jngrad is interested

[fer-rum]

The EOSC SQAaaS project actually has a set of badges they give out based on software quality criteria.

The project itself and this badging might be worth mentioning in the paper as an already existing approach.

@CaptainSifff WDYT?

[CaptainSifff]

So first, let's notify interested people: @jlinx , @jngrad

[CaptainSifff]

It might be worth mentioning, but I'm a bit hesitant on how to include it. These badges are Software project specific, but not necessarily attached to a person (which is the notion I'd like to have for this paper). We should get graduates that are qualified to produce software that gets these badges. @jngrad has worked a lot on badging in the-teachingRSE-project/competencies#33 . How about you have a look how it fits in there?

[jngrad]

We should probably start with the digital badges handbook (Ifenthaler et al. 2016, doi:10.1007/978-3-319-15425-1) and decide on the direction the badging discussion should take. There is already abundant literature on the subject in the healthcare field, yet surprisingly little in the CSE field. There is a concrete example in the Carpentries, where their initial objectives weren't met and the badges were repurposed, and in the predecessor of ACCESS (not sure if ACCESS itself still uses them).

Here are a few topics that emerged from my bibliographic search:

• digital badges for learners
  • incentives (viral effect, completionists, peer pressure) and gamification
  • micro-credentials (academic recognition for employees in a very narrow field, alternative to ECTS for students, unclear value for hiring managers)
  • negative aspects (distraction, frustration from unclear goals)
• digital badges for teachers
  • example of Carpentries: instructors must earn a badge to organize a workshop or vote
  • micro-credentials
• digital badges for source code
  • code quality (criterion for RSE grant proposals)
  • code coverage (criterion for RSE grant proposals)
  • availability in the cloud (Binder, Gitpod, etc.)
  • availability in package managers (npm, pypi, easybuild, linux OSes, etc.)
• digital badges for organizations
  • example of CHAOSS badges for code of conduct (conferences, workshops, code repositories)
• who should own, mint and deliver badges
  • corporate badges (e.g. GitHub) vs community badges (e.g. Fedora) vs institution badges (e.g. Open Shields)
  • badge value is tied to the organization that issues them

As discussed in the last meeting, I have paused the writing because I'm not sure in which direction we would like to go. It is also too detailed for the manuscript and should probably go to the appendix if we plan on writing more on the subject. Some topics are also out of scope for the manuscript, and having a separate manuscript would help us explore all aspects of digital badging, assuming we have something new to contribute in that field.

[knarrff]

ping for interest

[MakisH]

In my project, we now want to incentivize quality community contributions using guidelines and such quality level badges:

https://precice.discourse.group/t/shape-the-future-of-the-precice-ecosystem-the-preeco-project/2019

We are actually lacking a bit the incentivization aspect at the moment. If you have any ideas, please also comment there.

(this blog post should eventually have a Zenodo DOI, we are starting to experiment with a suggestion by @jngrad)

[jngrad]

My views on micro-credentials have shifted a bit in the last 12 months. I am now slightly more on the skeptical side, when it comes to certifying personal skills. For example, when recruiting personnel for a simulation or coding position, my organization systematically conducts a short coding interview, even when programming certificates are attached to the CV. We also review the applicant code contributions on GitHub, GitLab or other code repositories. I'm not sure badges would eliminate the need for a coding interview during the hiring process.

I still see value in micro-credentials for B.Sc./M.Sc./Ph.D. students due to the motivating effects of gamification, although the lack of standardization and interoperability leads to fragmentation. For example, some MOOC platforms issue PDF certificates that are tied to their brand, making it difficult to compare them against certificates of other MOOC platforms. There are European initiatives to issue standardized micro-credentials (Erasmus+ ERASMUS-EDU-2024-POL-EXP-MICRO-CRED, EuroHPC Joint Undertaking virtual training academy certification DIGITAL-EUROHPC-JU-2023-ACADEMY-02-01), which in the case of Erasmus+ has attracted hundreds of proposals.

To add some nuance to my previous statement, the non-interoperable nature of currently available badge infrastructures is not necessarily a problem per se. Micro-credentials can be useful even when intimately tied to a project community, like Fedora or preCICE, since members of these communities see the minter as an authority. In addition, some badging systems like the one developed by preCICE require human input, in the form of a review of the source code. In this case, there is an additional barrier-to-entry, since getting badges minted on an interoperable infrastructure would require you to write a bot that can parse human input and communicate it to the infrastructure. This is doable, if you have the patience to write such a bot. For example, JOSS requires human input during the initial search for reviewers / desk reject stage; once they find a reviewer who declares the manuscript to be in scope with the journal, the editor-in-chief assigns the issue to the reviewer, and a bot that listens to GitHub events triggers the creation of a review PR using the manuscript branch url's in the issue, and mints a DOI and the corresponding badge.

I know of one HPC facility that went into the direction of a qualification exam: ARCHER2 Driving Test, which was launched in 2022. I'm not sure if they were influenced by the HPC Certification Forum (HPC-CF). The HPC facility at TU Dresden provides links to individual skills in the HPC-CF skill tree when listing learning objectives and pre-requisites in a syllabus, e.g. Performance analysis of HPC applications with PIKA.