Difference between software engineering and writing research software

[richfitz]

Most of the tools that we use to increase reproducibility (make, git, docker, vagrant, etc) are directly taken from software engineering fields, but we don't really consider how appropriate they are for use in research. I think that research software is has different issues to traditional software, though there is of course a lot of overlap.

I've seen this writing maker vs using make for workflows: stringing together existing, reliable, tools is a different proposition to stringing together bits of scientific software that are always changing (see notes here).

Git is great for versioning code, but not so great for dealing with bits of data that go along with the code, simulation outputs, GIS layers, etc. It remains an unsolved problem how to associate these bits together.

Virtual machines are most useful when dealing with relatively stable software. I don't think we have clear workflows for baking in the unchanging bits with the "research bits" that never have a stable version (I'm currently dealing with this on a collaborative project with software engineers).

Research software is different to traditional software: for most scientists, the "final version" is designed to be run only once. I'm interested in working out where tools from software engineering are a good fit and where research-specific tooling is required. Our our problems actually different, or is this just a special case of Special Snowflake Syndrome?

[dmlond]

I think this goes along with this issue. The software carpentry movement is all about helping researchers understand the importance of using good software engineering practices to make their software reuseable. If researchers are not using these practices when writing code, they should be educated and encouraged to follow a path to where they are using them. One of the products of this hackathon would be to develop these paths.

[cboettig]

I think this is a great topic (i.e. "where can existing tooling be adapted to (long-tail) academic research, and where do we need new tools because nothing that exists is appropriate").

However, I think it probably needs to be reframed from the title given to be most productive. Software engineering practices aren't any more uniform than academic research software practices. There's big software engineering and little software engineering, and industry doesn't offer a monolithic set of 'best practices' but rather dozens of different & contradictory models (just as academia does).

There are many researchers for whom software isn't something that is designed to be 'run once' but rather something that other people have written for them and they use without writing a single line of code. I've spoken to some very happy users of Galaxy, for instance, where users can just click together pieces of robust software without special training and get reproducible workflows. But in my own field (where researchers tend to have some suite of local software already installed in which they know how to do what they need to do), it is my impression that such monolithic online workflow systems have largely failed.

We do have mechanisms for baking unchanging / stable bits with research bits; e.g. isn't any script that imports a library (that in turn runs on an os etc) combining a research bit with lots of increasingly stable bits? Of course there are many problems with how this works (or doesn't work).

My own intuition is that the greatest problem is not the one at the level that SWC targets (novices being introduced things like python, git, and using functions), but is most severe at the level of those who are already writing research software with the intention of it being reused, but being written at the scale of the single lab or single individual. This group may most lack either the tools or practices needed to do this efficiently in the academic context. Maybe that's not the right group to focus on, but I think we need to be explicit about what kind of academic research we're talking about to make the most progress on this topic. (didn't mean this comment to sound critical, I think there's a lot we can talk about here!)