Reviewer 1

[labarba]

Recommendation: Accept With No Changes

Comments: I have no major criticism of the article and would like to instead offer just some small comments and suggestions for places where the scope could be broadened or shifted somewhat. But I don't consider any of my comments crucial.

• [x] I find the discussion of different open-source software licenses somewhat tangential to the overall flow of the article. In particular, the differences between GPL-style and BSD-style licenses ultimately don't matter for the main point of the article, in my opinion.
• [x] Even more specifically, unless the intention is to align license choices with more general political orientation (i.e., standard left vs. right government politics), I think that the word "political" (page 4, line 22) may be confusing. Maybe "ideological" better conveys the intention here? If the intention is to convey an alignment with standard political orientation, that should be made more explicit. I imagine that copyleft might be considered more "leftist" and BSD-style license -- being business-friendly -- might be considered more "right-wing / neoliberal".
• [x] Regarding the relationship between "reproducible" and "reusable": an interesting perspective is offered by Gael Varoquaux in a blog post he wrote about this topic: http://gael-varoquaux.info/programming/software-for-reproducible-science-lets-not-have-a-misunderstanding.html . This raises some question about the conclusion reached at the end of the section "Open-source software in research" -- suggesting instead that there is a partial overlap between reproducibility and reusability, but that these are not identical and that prioritizing one over the other (Varoquaux clearly prefers reusability) guides the activities that a research team may pursue. Debating this perspective might add some interesting complexity.
• [x] If the scope allows, it might also be interesting to discuss some of the challenges/risks of openness. For example, researchers in climate science, who are under threat of having their research scrutinized in bad faith by climate change-denying "merchants of doubt".

Additional Questions:

1. How relevant is this manuscript to the readers of this periodical? Please explain your rating in the Detailed Comments section.: Don’t know the readership

2. Please summarize what you view as the key point(s) of the manuscript and the importance of the content to the readers of this periodical. If you don't have any comments, please type No Comments.: This paper links the praxis of open source software with the goals of computational reproducibility.

3. Is the manuscript technically sound? Please explain your answer in the Detailed Comments section.: Yes

4. What do you see as this manuscript's contribution to the literature in this field?: Using Winograd and Flores concept of connectivity, the article reaches the conclusion that reproducibility is not primarily a technical issue, but a socio-technical one, revolving around creating and maintaining trust through conversation. Along the way, the paper does nicely in providing an explanation of some of the basic tenets of open-source software development and the mechanics of the conversations that underlie these mechanics, and at the same time in providing a framework for thinking about how these mechanics create a particular substrate for reproducible research, especially within computational science.

5. What do you see as the strongest aspect of this manuscript?: Novel framework for thinking about the links between open-source software and broader computational reproducibility.

6. What do you see as the weakest aspect of this manuscript?:

7. Does the manuscript contain sufficient and appropriate references? Please elaborate in the Detailed Comments section.: References are sufficient and appropriate

8. Does the introduction state the objectives of the manuscript in terms that encourage the reader to read on? Please explain your answer in the Detailed Comments section.:

9. How would you rate the organization of the manuscript? Please elaborate in the Detailed Comments section.:

10. Is the length of the manuscript appropriate for the topic? Please elaborate in the Detailed Comments section.: Satisfactory

11. Please rate the readability of this manuscript in the Detailed Comments section.: Easy to read

Please rate the manuscript. Explain your choice in the Detailed Comments section.: Excellent

[labarba]

• I have moved the text on the GPL-style vs. BSD-style licenses to a sidebar, and edited it for conciseness. I agree with the reviewer that it is tangential to the main argument, but I decided to leave it as a sidebar because of the widespread confusion about these terms.

[labarba]

• the word "political" was swapped with "ideological" as suggested

[labarba]

This raises some question about the conclusion reached at the end of the section "Open-source software in research" -- suggesting instead that there is a partial overlap between reproducibility and reusability

I'm unclear about the point the referee is making here. The third paragraph of that section in the manuscript makes the point that transparency (making code available) is not sufficient for reproducibility, and that the ability of other researchers to confirm the results with an original author's artifacts also requires that the code be (at least somewhat) reusable.

Gael's post comments on the fact that a lot of code released to accompany a paper is prototype quality and not built for easy maintenance or reuse: "prototypes are designed to be thrown away." Then he says that in his opinion "quality is indeed central to doing science with code" and "Good scientific code is code that can be reused"—which I interpret to be in agreement with my suggestion that transparency as proxy for reproducibility is not sufficient. Gael later emphasizes that we also do not want to focus too early on high quality, which comes at a cost. He then says that "Reuse is very difficult and cannot be a requirement for all publications." But here he means reuse as "applying the analysis to different problems or data," whereas for reproducibility we require a weaker form of reuse: in the context of the original publication to confirm the results.

[labarba]

• I added a short clarification about the meaning of "reuse" used here versus the stronger meaning used by Gael.

[labarba]

discuss some of the challenges/risks of openness

Thank you for the suggestion, but I have reached the article length limits and this topic seems a little tangential to my argument.