search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
720 stars 38 forks source link

[REVIEW]: Fast and flexible simulation and parameter estimation for synthetic biology using bioscrape #5057

Closed editorialbot closed 1 year ago

editorialbot commented 1 year ago

Submitting author: !--author-handle-->@ayush9pandey<!--end-author-handle-- (Ayush Pandey) Repository: https://github.com/biocircuits/bioscrape/ Branch with paper.md (empty if default branch): joss-paper Version: v1.2.2 Editor: !--editor-->@csoneson<!--end-editor-- Reviewers: @Farnazmdi, @Robaina Archive: 10.5281/zenodo.7677726

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/5935db21ffab5b33069d05b7adbdf094"><img src="https://joss.theoj.org/papers/5935db21ffab5b33069d05b7adbdf094/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/5935db21ffab5b33069d05b7adbdf094/status.svg)](https://joss.theoj.org/papers/5935db21ffab5b33069d05b7adbdf094)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@Farnazmdi & @Robaina, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @csoneson know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @Robaina

📝 Checklist for @Farnazmdi

editorialbot commented 1 year ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 1 year ago 
Software report:

github.com/AlDanial/cloc v 1.88  T=0.26 s (434.6 files/s, 152806.2 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Cython                          11           2122           1985           7013
XML                             47             76              3           6935
Python                          24            628           1060           3359
Jupyter Notebook                24              0          12927           2642
TeX                              1             66              0            615
Markdown                         2             56              0            146
YAML                             4              6              4             86
-------------------------------------------------------------------------------
SUM:                           113           2954          15979          20796
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository
editorialbot commented 1 year ago

Wordcount for paper.md is 1520

editorialbot commented 1 year ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1109/tcbb.2011.63 is OK
- 10.1101/000885 is OK
- 10.1126/science.1179047 is OK
- 10.1002/chin.200630297 is OK
- 10.1002/anie.201409847 is OK
- 10.1098/rsfs.2011.0047 is OK
- 10.1073/pnas.1200161109 is OK
- 10.1038/nmeth.2794 is OK
- 10.1038/nature11897 is OK
- 10.1186/1471-2105-10-343 is OK
- 10.1007/s11222-007-9043-x is OK
- 10.1049/iet-syb.2010.0013 is OK
- 10.1093/bioinformatics/btt380 is OK
- 10.1063/1.3664126 is OK
- 10.1038/nature09326 is OK
- 10.1007/978-3-642-22110-1_8 is OK
- 10.3182/20140824-6-za-1003.02771 is OK
- 10.1109/acc.2016.7524946 is OK
- 10.1038/nrmicro2056 is OK
- 10.1006/jmbi.1993.1092 is OK
- 10.1036/1097-8542.yb130009 is OK
- 10.23943/princeton/9780691161532.001.0001 is OK
- 10.1109/mcse.2010.118 is OK
- 10.1371/journal.pone.0079345 is OK
- 10.1073/pnas.1013171108 is OK
- 10.1038/nprot.2011.432 is OK
- 10.1109/tbcas.2016.2526668 is OK
- 10.1126/science.aad8559 is OK
- 10.1073/pnas.1202344109 is OK
- 10.1371/journal.pcbi.1009987 is OK
- 10.1002/rnc.6013 is OK
- 10.1101/029967 is OK
- 10.1046/j.1365-2958.2002.02891.x is OK
- 10.1016/j.jmb.2003.09.082 is OK
- 10.2140/camcos.2010.5.65 is OK
- 10.1016/j.cell.2008.01.027 is OK
- 10.1186/1754-1611-2-5 is OK
- 10.1101/2022.10.05.511053 is OK
- 10.1038/nbt.2401 is OK

MISSING DOIs

- None

INVALID DOIs

- None
csoneson commented 1 year ago

👋🏼 @ayush9pandey, @Farnazmdi, @Robaina - this is the review thread for the submission. All of our communications will happen here from now on.

Please check the post at the top of the issue for instructions on how to generate your own review checklist. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues directly in the software repository. If you do so, please mention this thread so that a link is created (and I can keep an eye on what is happening). Please also feel free to comment and ask questions in this thread. It is often easier to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

Please feel free to ping me (@csoneson) if you have any questions or concerns. Thanks!

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

Robaina commented 1 year ago

Review checklist for @Robaina

Conflict of interest

Code of Conduct

General checks

Functionality

As far as I can tell, bioscrape does provide the functionality described in the manuscript. The repo contains examples for the described functionalities.

The authors claim that bioscrape runs faster than the current alternatives:

"Some popular software packages that do somewhat similar tasks as the bioscrape package are MATLAB’s SimBiology toolbox (MATLAB, 2016), Stochpy (Maarleveld, 2013), COPASI (Hoops et al., 2006), and Tellurium (Medley et al., 2018). However, the bioscrape package is faster [...] "

I think that such a claim should be backed by some sort of performance benchmark test.

Documentation

Authors include extensive examples as Jupyter Notebooks. One of them was broken, but it's been taken care of in this PR:

There are guidelines. Contributing guidelines were somewhat scarce (i.e. no mention how to set a developer environment or directions on how to contribute). However, authors will add a "CONTRIBUTING.md" file to expand a bit these guidelines. This is been addressed in this PR

Software paper

It does have a statement of need. Perhaps the target audience could be made a bit more specific, i.e. bioscrape i built for researchers in "field, perhaps also educational?. An issue has been raised in the repo.

Robaina commented 1 year ago

Hi @sclamons,

I'm one of the reviewers of this submission (submitted by @ayush9pandey). I'm currently following JOSS guidelines regarding authorship. I see that you have made a substantial amount of commits to the repo but I cannot map your username to any name in the author list of the submitted manuscript. Are you one of the authors listed in the manuscript? Thank you!

Robaina commented 1 year ago

Hi @csoneson,

this is my first review in JOSS, it's not entirely clear to me which questions/comments should I post here and which ones in the issues section of the repo. Are there any general guidelines for this? For instance, comments regarding the manuscript should be posted in this thread or in the repo? Thank you!

csoneson commented 1 year ago

Hi @Robaina - different reviewers have different preferences; in my experience it's often a good idea to post specific improvement suggestions and encountered problems as individual issues in the software repo, where they can be discussed in separate threads and it's easier to keep track of their status, and to keep this review issue more for general "summary" comments or questions. Comments regarding the manuscript can go in either place (as the manuscript also lives in the software repo).

sclamons commented 1 year ago

Hi Semidán,

I know which paper this is, but I can't actually see the submission (I graduated and left the lab about a year ago). My name is Sam Clamons, which you probably could have guessed if it were on the author list! I don't think I should be, particularly - I made relatively minor contributions using some unorthodox dev practices that caused me to submit a disproportionate number of commits.

Thanks for checking! Very much appreciated. Sam

On Sat, Jan 14, 2023, 2:21 AM Semidán Robaina, Ph.D. < @.***> wrote:

Hi @sclamons https://github.com/sclamons,

I'm one of the reviewers of this submission (submitted by @ayush9pandey https://github.com/ayush9pandey). I'm currently following JOSS guidelines regarding authorship https://joss.readthedocs.io/en/latest/review_criteria.html. I see that you have made a substantial amount of commits to the repo https://github.com/biocircuits/bioscrape/graphs/contributors but I cannot map your username to any name in the author list of the submitted manuscript. Are you one of the authors listed in the manuscript? Thank you!

— Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/5057#issuecomment-1382707915, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAM7DUY7VUIYPFIJRITZPEDWSJ43LANCNFSM6AAAAAATVXVKV4 . You are receiving this because you were mentioned.Message ID: @.***>

Robaina commented 1 year ago

Hi @csoneson ,

I now have completed the review of this submission. I have raised some issues in the bioscrape repo (all linked here). Authors are already taking care of two of them (I just raised the other two).

I have locally installed bioscrape and tested all provided examples. I find the package a nice addition to the community, it is well documented and easy to use (congratulations to the authors!)

There is only one mark that I left unchecked in the Review Checklist -> Performance. In the manuscript: Statement of Need, authors claim that bioscrape runs faster than competing methods (here probably I would also clarify in which functions it runs faster, i.e., parameter estimation?). I think that such a claim should be backed by a benchmark analysis, which I did not see in the manuscript or the repo.

ayush9pandey commented 1 year ago

Thanks @Robaina ! I assume you wanted to address that message to me and not Sam. So I am replying to confirm that I am working on the issues that you raised on the repository. For the performance claims on the paper, I totally agree with you and will make required edits to the paper.

Robaina commented 1 year ago

Hi @ayush9pandey, sorry for not tagging you. I wanted to address that message to both of you. Thanks, the repo and paper look very nice.

There is one minor thing I forgot to mention. The code you use in the example within "Statement of Need" in the paper is not runable. I guess because it corresponds to an older version of bioscrape. This is not that important in my opinion but just to let you know in case you want to update it.

ayush9pandey commented 1 year ago

Thanks for bringing that up @Robaina ! We want to ensure backward compatibility as much as possible, so will ensure that the example in the paper works. I believe it is just a minor typo in the code in the figure.

Robaina commented 1 year ago

Thanks @Robaina ! I assume you wanted to address that message to me and not Sam. So I am replying to confirm that I am working on the issues that you raised on the repository. For the performance claims on the paper, I totally agree with you and will make required edits to the paper.

Right, I messed up the tags, sorry, corrected now.

Robaina commented 1 year ago

Hi @csoneson ,

I now have checked all marks. Authors have addressed all the issues I raised (including the benchmark test on performance) and I don't have any more issues with bioscrape. I think it's ready for publication.

Thanks for inviting me to review this submission.

csoneson commented 1 year ago

Thank you very much @Robaina!

csoneson commented 1 year ago

@Farnazmdi - could you update us on your review? Don't hesitate to reach out if you have questions. Thanks!

Farnazmdi commented 1 year ago

@csoneson sorry for the late reply, I was super busy. I am just starting the review and will update you soon.

csoneson commented 1 year ago

@Farnazmdi - no worries, thank you!

Farnazmdi commented 1 year ago

Review checklist for @Farnazmdi

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

ayush9pandey commented 1 year ago

@editorialbot generate pdf

ayush9pandey commented 1 year ago

Just want to generate the updated PDF after Robaina's review.

editorialbot commented 1 year ago

:warning: An error happened when generating the pdf.

ayush9pandey commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

Farnazmdi commented 1 year ago

Hi @csoneson and @ayush9pandey I finished my review of bioscrape. I believe it is a nice package with useful capabilities to people in the field. I left a few minor comments in the issues with "JOSS Review -- ..." in the titles. As soon as those are addressed I think it is ready!

Thank you for inviting me to review this package!

csoneson commented 1 year ago

Thanks @Farnazmdi!

ayush9pandey commented 1 year ago

Thanks @Farnazmdi for reviewing bioscrape! I am working on the issues that you raised, but this week is more busy than usual, so, I am planning to finish the changes by next week.

ayush9pandey commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

Farnazmdi commented 1 year ago

@ayush9pandey Thank you for addressing my comments regarding your package and manuscript. I believe with your latest changes all my concerns are resolved and it is ready to be published!

csoneson commented 1 year ago

Thanks @Farnazmdi!

@ayush9pandey - as both reviewers have now given their approval, I will also take a quick look through the submission and get back to you shortly with the next steps.

csoneson commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

csoneson commented 1 year ago

@editorialbot check references

editorialbot commented 1 year ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1109/tcbb.2011.63 is OK
- 10.1101/000885 is OK
- 10.1126/science.1179047 is OK
- 10.1002/chin.200630297 is OK
- 10.1002/anie.201409847 is OK
- 10.1098/rsfs.2011.0047 is OK
- 10.1073/pnas.1200161109 is OK
- 10.1038/nmeth.2794 is OK
- 10.1038/nature11897 is OK
- 10.1186/1471-2105-10-343 is OK
- 10.1007/s11222-007-9043-x is OK
- 10.1049/iet-syb.2010.0013 is OK
- 10.1093/bioinformatics/btt380 is OK
- 10.1063/1.3664126 is OK
- 10.1038/nature09326 is OK
- 10.1007/978-3-642-22110-1_8 is OK
- 10.3182/20140824-6-za-1003.02771 is OK
- 10.1109/acc.2016.7524946 is OK
- 10.1038/nrmicro2056 is OK
- 10.1006/jmbi.1993.1092 is OK
- 10.1036/1097-8542.yb130009 is OK
- 10.23943/princeton/9780691161532.001.0001 is OK
- 10.1109/mcse.2010.118 is OK
- 10.1371/journal.pone.0079345 is OK
- 10.1073/pnas.1013171108 is OK
- 10.1038/nprot.2011.432 is OK
- 10.1109/tbcas.2016.2526668 is OK
- 10.1126/science.aad8559 is OK
- 10.1073/pnas.1202344109 is OK
- 10.1371/journal.pcbi.1009987 is OK
- 10.1002/rnc.6013 is OK
- 10.1101/029967 is OK
- 10.1046/j.1365-2958.2002.02891.x is OK
- 10.1016/j.jmb.2003.09.082 is OK
- 10.2140/camcos.2010.5.65 is OK
- 10.1016/j.cell.2008.01.027 is OK
- 10.1186/1754-1611-2-5 is OK
- 10.1101/2022.10.05.511053 is OK
- 10.1038/nbt.2401 is OK

MISSING DOIs

- 10.1093/bioinformatics/btv363 may be a valid DOI for title: libRoadRunner: a high performance SBML simulation and analysis library

INVALID DOIs

- None
csoneson commented 1 year ago

@ayush9pandey - I went through the submission and it looks good; I made a small PR with some fixes to the bibliography, please check and merge if you agree. You can then generate a new proof with @editorialbot generate pdf.

Then, the next steps are as follows:

I can then move forward with accepting the submission.

ayush9pandey commented 1 year ago

Thanks @csoneson ! I merged the PR and tagged the release for v1.2.2 on Github: https://github.com/biocircuits/bioscrape/releases/tag/v1.2.2. I submitted the archive to Zenodo: 10.5281/zenodo.7677726 https://zenodo.org/record/7677726

csoneson commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

csoneson commented 1 year ago

@editorialbot check references

editorialbot commented 1 year ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1093/bioinformatics/btl485 is OK
- 10.1093/bioinformatics/btac770 is OK
- 10.1093/bioinformatics/btv363 is OK
- 10.1371/journal.pcbi.1006220 is OK
- 10.25080/majora-92bf1922-00a is OK
- 10.1016/j.cell.2012.01.002 is OK
- 10.1109/tcbb.2011.63 is OK
- 10.1101/000885 is OK
- 10.1126/science.1179047 is OK
- 10.1002/chin.200630297 is OK
- 10.1002/anie.201409847 is OK
- 10.1098/rsfs.2011.0047 is OK
- 10.1073/pnas.1200161109 is OK
- 10.1126/science.1105809 is OK
- 10.1038/nmeth.2794 is OK
- 10.1038/nature11897 is OK
- 10.1186/1471-2105-10-343 is OK
- 10.1007/s11222-007-9043-x is OK
- 10.1049/iet-syb.2010.0013 is OK
- 10.1093/bioinformatics/btt380 is OK
- 10.1063/1.3664126 is OK
- 10.1038/nature09326 is OK
- 10.1007/978-3-642-22110-1_8 is OK
- 10.3182/20140824-6-za-1003.02771 is OK
- 10.1109/acc.2016.7524946 is OK
- 10.1038/nrmicro2056 is OK
- 10.1006/jmbi.1993.1092 is OK
- 10.1036/1097-8542.yb130009 is OK
- 10.23943/princeton/9780691161532.001.0001 is OK
- 10.1109/mcse.2010.118 is OK
- 10.1371/journal.pone.0079345 is OK
- 10.1073/pnas.1013171108 is OK
- 10.1101/071100 is OK
- 10.1038/nprot.2011.432 is OK
- 10.1109/tbcas.2016.2526668 is OK
- 10.1126/science.aad8559 is OK
- 10.1073/pnas.1202344109 is OK
- 10.1371/journal.pcbi.1009987 is OK
- 10.1002/rnc.6013 is OK
- 10.1101/029967 is OK
- 10.1046/j.1365-2958.2002.02891.x is OK
- 10.1016/j.jmb.2003.09.082 is OK
- 10.2140/camcos.2010.5.65 is OK
- 10.48550/arXiv.1202.3665 is OK
- 10.1016/j.cell.2008.01.027 is OK
- 10.1186/1754-1611-2-5 is OK
- 10.1101/2022.10.05.511053 is OK
- 10.1038/nbt.2401 is OK

MISSING DOIs

- None

INVALID DOIs

- None
csoneson commented 1 year ago

Thanks @ayush9pandey. It looks like there's still one DOI missing (for COPASI), which is likely (I notice now) because there are two entries in the .bib file for this paper (and only one has the DOI). Could you take a look and clean up the .bib file to only contain the references that are actually cited in the paper, and remove any duplicated entries? Thanks!

ayush9pandey commented 1 year ago

Sorry about that. I have cleaned up the references now.

ayush9pandey commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

ayush9pandey commented 1 year ago

@editorialbot check references

editorialbot commented 1 year ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1093/bioinformatics/btl485 is OK
- 10.1093/bioinformatics/btac770 is OK
- 10.1093/bioinformatics/btv363 is OK
- 10.1371/journal.pcbi.1006220 is OK
- 10.25080/majora-92bf1922-00a is OK
- 10.1016/j.cell.2012.01.002 is OK
- 10.1109/tcbb.2011.63 is OK
- 10.1098/rsfs.2011.0047 is OK
- 10.1073/pnas.1200161109 is OK
- 10.1126/science.1105809 is OK
- 10.1186/1471-2105-10-343 is OK
- 10.1109/mcse.2010.118 is OK
- 10.1371/journal.pone.0079345 is OK
- 10.1073/pnas.1013171108 is OK
- 10.1101/071100 is OK
- 10.1002/rnc.6013 is OK
- 10.48550/arXiv.1202.3665 is OK
- 10.1021/acssynbio.2c00534 is OK

MISSING DOIs

- None

INVALID DOIs

- None