LBM SOP: Artifacts Recommended, but not observed

[smiths]

@peter-michalski, to write Section 5 of the paper, I'm adding a comparison of LBM observed artifacts to artifacts recommended for scientific software. The following artifacts appear on the recommended list, but not in our Table 7. In some cases though that is probably because you weren't really looking for the given artifact, not necessarily because it isn't present. Have you observed any of the following in LBM repos:

• Roadmap
• Code of conduct, policies for team members
• Citation
• Programming style guide
• description of team processes
• peer review
• checklists
• linters
• uninstall

[peter-michalski]

Roadmap - I did see this mentioned, but no specific artifacts for it. ("rare" mentions)

Code of conduct, policies for team members - do not recall ever seeing this

Citation - definitely saw this several times ("less common")

Programming style guide - I did see this as part of developers guide ("rare")

description of team processes - I do recall seeing "workflow" detailed ("rare"). Not sure if this is similar.

peer review - This was mentioned a few times in the interviews, but I do not recall seeing artifacts for this.

checklists - I did see this a few times ("rare")

linters - I did see this ("rare")

uninstall - I do not recall seeing this

[smiths]

Thank you @peter-michalski. I won't modify your original list of artifacts (Table 7). You came up with that list following a process, and the data it is based on is reflected in your measurement template. I wouldn't want to mess with that. My goal for asking you for additional information is to make sure that we don't highlight an artifact that does occur in LBM software repos.

I think I can safely mention the rare items from above as being an area for improvement in LBM software to match recommended practice. I'll make a point of saying that the items may occur rarely in the projects, but they weren't obvious enough to make our list of artifacts.

[smiths]

@peter-michalski, in your comment above you mentioned that you had seen checklists in the LBM repos, but only rarely. Is there any chance you know an example of a project that uses a checklist, and what the checklist is for? I'm writing about this now and it would strengthen our work if we could be a bit more specific.

[peter-michalski]

@smiths ESPResSo has a checklist for managing releases. There is also discussion in an issue in HemeLB about potentially creating a similar checklist for managing releases.

[smiths]

Great! That's exactly the kind of example I was looking for.