Are software papers the solution for measuring RSE productivity?

[mingfangwu]

Notes I took from the Vic f2f meeting:

• There is a spectrum of software, software papers shouldn't be the only solution
• Some software is useful tool but not that innovative, should software as such be published?
• Activity traffic is an indication of software contribution/productivity, github may be the solution.

[goldingn]

photos of flipchart from the Vic meetup:

[atruskie]

Attached are two workshop answers to this question from the QLD F2F (#19):

• overall: NO
• are the solution to what? recognition of contribution or attribution of contribution?
• having artefacts to track is useful
• but RSE publications (even for software papers) are not related to the actual work an RSE does
• from an HR perspective, there's no point producing software papers unless it can be counted as part of your job performance
• from an academic perspective, software papers could make us look like "pretend academics"
• Academic publishing is broken anyway, why do we want to conform to that model?
• One positive of software papers is that they require software to have documentation, to be maintainable, to have a DOI, and to have the software published --- these are all good things for open and reproducible science

• it encourages open source software - d/c, contribution measures
• The JOSS was mentioned
• Perhaps consulting models were a better fit than software papers?
• Again the spectrum between academic and professional has relevance
• Generally though, Yes, they are useful, gives us more KPIs

[davidbenncsiro]

One bullet point above is: • but RSE publications (even for software papers) are not related to the actual work an RSE does

Would it be useful to solicit examples of software related publications in order to assess the relevance of such artefacts to the work of a RSE? In some cases, the publication of the software itself and related papers or conference talks or posters may be highly relevant.

Also, rather than publications, perhaps revision control repositories containing non-trivial (by some definition of non-trivial) contents may be a more useful metric of productivity than publications, since the absence of a DOI is not necessarily an indicator of the utility of software.

[GerryRyder]

Blog post by Stephan Druskat, Daniel S. Katz, David Klein et al

"Credit and recognition for research software: Current state of practice and outlook"