[REVIEW]: Ngesh: a Python library for synthetic phylogenetic data

whedon commented 3 years ago

Submitting author: @tresoldi (Tiago Tresoldi) Repository: https://github.com/tresoldi/ngesh Version: 1.0 Editor: @majensen Reviewer: @DavidNickle, @rvosa Archive: 10.5281/zenodo.5552605

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/33b21b92804433dd0f5e61bacc323130"><img src="https://joss.theoj.org/papers/33b21b92804433dd0f5e61bacc323130/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/33b21b92804433dd0f5e61bacc323130/status.svg)](https://joss.theoj.org/papers/33b21b92804433dd0f5e61bacc323130)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@DavidNickle & @rvosa, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @majensen know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @DavidNickle

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@tresoldi) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @rvosa

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@tresoldi) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @DavidNickle, @rvosa it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 3 years ago

Failed to discover a Statement of need section in paper

whedon commented 3 years ago

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.17 s (180.3 files/s, 29345.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
XML                              2              6             16           1084
Python                          10            318            703            974
Markdown                         6            172              0            736
reStructuredText                 5            157            294            353
TeX                              1             11              0            112
YAML                             3              8              4             49
JSON                             1              0              0             23
make                             1              4              7              9
Bourne Shell                     2              1              0              5
-------------------------------------------------------------------------------
SUM:                            31            677           1024           3345
-------------------------------------------------------------------------------

Statistical information for the repository '7e95ab623396fc53f9a2293c' was
gathered on 2021/04/15.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
NicolaDM                         1           183              1            2.90
Tiago Tresoldi                  90          3986           2173           97.10

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
NicolaDM                    122           66.7          1.9               48.36
Tiago Tresoldi             1873           47.0          8.3               23.01

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

whedon commented 3 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1126/science.283.5406.1310 is OK
- 10.1093/sysbio/syr029 is OK
- 10.1093/molbev/msw046 is OK

MISSING DOIs

- 10.2307/1909876 may be a valid DOI for title: The elements of stochastic processes with applications to the natural sciences
- 10.1093/bioinformatics/btu181 may be a valid DOI for title: geiger v2.0: an expanded suite of methods for fitting macroevolutionary models to phylogenetic trees
- 10.1093/bioinformatics/bty633 may be a valid DOI for title: ape 5.0: an environment for modern phylogenetics and evolutionary analyses in R
- 10.1371/journal.pone.0180908 may be a valid DOI for title: BEASTling: A software tool for linguistic phylogenetics using BEAST 2

INVALID DOIs

- None

DavidNickle commented 3 years ago

Conflict of interest: I confirm that I have read the JOSS conflict of interest policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review. Code of Conduct

Repository: The source code for this software is available at the repository url YES

License: The repository contain a plain-text LICENSE file with the contents of an OSI approved software license. YES

Contribution and authorship: The submitting author made major contributions to the software and the full list of paper authors seem appropriate and complete. YES

Functionality: The software is functional and important adding very important knowledge to the public domain. YES

Installation: The installation procedure follows the documentation clearly. I performed many of the examples in the How To Use section of the manuscript. All worked as advertised

Functionality: I confirmed the functional claims of the software as well as some of my own.

Performance: It is pretty quick!

A statement of need: The authors have a clear audience in they have mind and the paper and the software should be readably available to that audience. It is potentially most useful for developing nulls in the evolution of languages

Installation instructions: I installed through PIP without any issues

Example usage: I went through the How To Use section. Everything worked.

Automated tests: YES — tests are available

Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support. YES

Software paper:

Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided? YES

A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is? YES

State of the field: Do the authors describe how this software compares to other commonly-used packages? YES Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)? YES References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax? YES

@majensen I think this paper should be accepted. It is a nice piece of software that can be useful for those that are interested in the evolutionary of languages. It even allows for recombination (words passing from one language into another — can you use the word pajama or pyjama).

tresoldi commented 3 years ago

Thanks! I will add the DOIs when reviews are over. I can also add a short "Statement of need" if preferred (once more, after the first round of review is over).

whedon commented 3 years ago

:wave: @rvosa, please update us on how your review is going (this is an automated reminder).

whedon commented 3 years ago

:wave: @DavidNickle, please update us on how your review is going (this is an automated reminder).

DavidNickle commented 3 years ago

My review is complete on 14 days ago. See above

majensen commented 3 years ago

@rvosa - how is your progress on this review? Just let me know timing expectations if you can thanks!

rvosa commented 3 years ago

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.
- [x] I confirm that I read and will adhere to the JOSS code of conduct.
- [x] Yes: the source code for this software is available at the repository url?
- [x] Yes: the repository contains a plain-text (MIT) LICENSE file with the contents of an OSI approved software license?
- [x] Yes: the submitting author (@tresoldi) has made all but one of the commits. Their sole authorship therefore seems appropriate.
- [x] Yes: this submission meets the scope eligibility described in the JOSS guidelines
- [x] Yes: the installation proceeds more or less as outlined in the documentation, though I needed to use pip3 instead. I blame my messy Python configuration though.
- [x] Yes: the functional claims of the software have been confirmed. I type ngesh and I get a b/d tree.
- [x] There were no explicit performance claims but also no issues.
- [ ] Maybe: there is no explicit statement of need but the Background section in the PDF makes the case reasonably well. I do note that the presented functionality is available in many other toolkits as well and as such does not constitute a major contribution.
- [x] Yes: there is a clearly-stated list of dependencies in the form of a requirements.txt
- [x] Yes: the authors include examples of how to use the software
- [x] Yes: the core functionality of the software is documented to a satisfactory level (e.g., API method documentation)?
- [x] Yes: there are automated tests so that the functionality of the software can be verified?
- [x] Yes: there are clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support
- [x] Yes: a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience has been provided
- [ ] No: the paper has no section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is
- [ ] No: the authors do not describe sufficiently how this software compares to other commonly-used packages. Simulating trees and characters is available in numerous other tools, e.g. DendroPy, Mesquite, Bio::Phylo and others
- [x] Yes: the paper is well written (i.e., it does not require editing for structure, language, or writing quality)
- [ ] Maybe: the list of references is complete in that it has the citations in the manuscript, but those are incomplete, as noted above.

danielskatz commented 3 years ago

@rvosa - rather than adding a new comment with a new checklist, please just use your checklist in the first comment in this issue.

majensen commented 3 years ago

@tresoldi - @rvosa would like to see some text comparing your package to other available tools, and to highlight how your software meets an existing need. JOSS does requirement this so-called "Statement of Need". Can you work to add this? thanks

tresoldi commented 3 years ago

Sure, will do!

tresoldi commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

tresoldi commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

tresoldi commented 3 years ago

@majensen I have finished reworking the draft. The last paragraph of the "Background" was reworked into a "Statement of need", with no real changes in what is said. I also removed some filler words from the rest of the draft to make sure it stays within 3 pages when compiled, but contents are otherwise unchanged.

majensen commented 3 years ago

Thanks @tresoldi - I apologize for being out of the loop for a while. I will make every effort to review the current version this weekend.

tresoldi commented 3 years ago

No problem, it also took me longer than expected to address the "statement of need" issue. Thanks, @majensen !

majensen commented 3 years ago

Sorry for the continued delay @tresoldi -- finally able to have a look.

What I appreciate about the paper is the idea of "technology transfer" of phylogenetic methods from biology to linguistics. (Although, I think biology imported the basic ideas from comparative linguistics a long long time ago!)

I have to agree with @rvosa, however, that there are many and extensive tools that have been developed for simulating trees in the evolutionary biology world - see https://evolution.genetics.washington.edu/phylip/software.html#Simulation for an old list. Now, the fact that there are about 30 listed there suggests that people need certain things for their applications that aren't quite there in other programs, so I am not inclined to dismiss your submission just because others have programs (in other languages or with more specialized algorithms) that perform similar functions.

However - I would like a little more discussion comparing your application with a few of these common evolutionary biology tools. Can I ask you to review the ones suggested by @rvosa (DendroPy, Mesquite, and Bio::Phylo) and compare your application with these. I'm not too concerned with extending the length of the paper. I would also suggest briefly outlining a real workflow in the linguistic space, one that you have used ngesh for, perhaps. This would be helpful to readers in that discipline and I think it would help argue for the usefulness of your contribution.

Sorry to make more work after such a long delay, but I think you'll have a better paper at the end. Plus, we have @rvosa to convince! Thanks for your consideration -

majensen commented 3 years ago

@tresoldi - checking in on your progress, thanks

majensen commented 3 years ago

@tresoldi - please let me know how things are going - if you prefer, feel free to contact me via email - maj -dot- fortinbras -at- gmail -dot- com.

majensen commented 3 years ago

:wave: @tresoldi - hoping to hear from you soon; thanks!

tresoldi commented 3 years ago

@majensen sorry I have been so much out of touch. I am able to get back to it now, I have now planned to submit the reviewed draft by tomorrow (Friday) evening.

tresoldi commented 2 years ago

@whedon generate pdf

whedon commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

tresoldi commented 2 years ago

@majensen finally done it, I have sent you an email with a short explanation as well. Sorry for the delay.

majensen commented 2 years ago

Sorry all, I have been AWOL. I hope to return to this within the week.

majensen commented 2 years ago

@whedon check references

whedon commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1126/science.283.5406.1310 is OK
- 10.1093/sysbio/syr029 is OK
- 10.1093/molbev/msw046 is OK

MISSING DOIs

- 10.1038/sdata.2018.205 may be a valid DOI for title: Cross-Linguistic Data Formats, advancing data sharing and re-use in comparative linguistics
- 10.2307/2333730 may be a valid DOI for title: The elements of stochastic processes with applications to the natural sciences
- 10.1126/science.1219669 may be a valid DOI for title: Mapping the origins and expansion of the Indo-European language family
- 10.16993/baj.h may be a valid DOI for title: The Digital Revolution in Scholarly Editing
- 10.1093/bioinformatics/btu181 may be a valid DOI for title: geiger v2.0: an expanded suite of methods for fitting macroevolutionary models to phylogenetic trees
- 10.1093/bioinformatics/bty633 may be a valid DOI for title: ape 5.0: an environment for modern phylogenetics and evolutionary analyses in R
- 10.1371/journal.pone.0180908 may be a valid DOI for title: BEASTling: A software tool for linguistic phylogenetics using BEAST 2

INVALID DOIs

- None

majensen commented 2 years ago

Hi @tresoldi - it is looking good to me. I would still like to see a few sentences describing a typical analysis in historical linguistics and where ngesh fits in to that analysis.

Can you update your paper.bib with the DOIs found by Whedon as MISSING DOIs above.

Also, can you update the reference to Harmon's book with more precise details as follows:

title: Phylogenetic Comparative Methods: Learning from Trees
year: 2018
isbn: 978-1719584463
publisher: CreateSpace Independent Publishing (not University of Idaho)

These details are from Amazon.

At the end of Line 26 in the PDF proof: can you replace "constrains" with "constraints".

thanks!

tresoldi commented 2 years ago

@whedon generate pdf

whedon commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

tresoldi commented 2 years ago

@whedon generate pdf

whedon commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

tresoldi commented 2 years ago

@whedon generate pdf

whedon commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

tresoldi commented 2 years ago

@whedon check references

whedon commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1038/sdata.2018.205 is OK
- 10.2307/2333730 is OK
- 10.1126/science.283.5406.1310 is OK
- 10.1093/sysbio/syr029 is OK
- 10.1093/molbev/msw046 is OK
- 10.1126/science.1219669 is OK
- 10.16993/baj.h is OK
- 10.1093/bioinformatics/btu181 is OK
- 10.1093/bioinformatics/bty633 is OK
- 10.1371/journal.pone.0180908 is OK
- 10.1371/journal.pcbi.1006650 is OK

MISSING DOIs

- 10.32942/osf.io/e3xnr may be a valid DOI for title: Phylogenetic Comparative Methods: Learning from Trees 

INVALID DOIs

- None

tresoldi commented 2 years ago

Sorry for spamming with whedon, but there is always a tiny detail missing... This should be the last round.

tresoldi commented 2 years ago

@whedon generate pdf

whedon commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

tresoldi commented 2 years ago

@whedon check references

whedon commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1038/sdata.2018.205 is OK
- 10.2307/2333730 is OK
- 10.1126/science.283.5406.1310 is OK
- 10.32942/osf.io/e3xnr is OK
- 10.1093/sysbio/syr029 is OK
- 10.1093/molbev/msw046 is OK
- 10.1126/science.1219669 is OK
- 10.16993/baj.h is OK
- 10.1093/bioinformatics/btu181 is OK
- 10.1093/bioinformatics/bty633 is OK
- 10.1371/journal.pone.0180908 is OK
- 10.1371/journal.pcbi.1006650 is OK

MISSING DOIs

- None

INVALID DOIs

- None

tresoldi commented 2 years ago

Thank you very much, @majensen ! I have updated accordingly, also with a small addition to the Statement of Need. You can compare the current source with the one you checked here: https://github.com/tresoldi/ngesh/compare/172d382..2561440

majensen commented 2 years ago

Ok @tresoldi, I'm happy with this. We are nearly there- can you please use Zenodo, FigShare or other service to create an open archive of the repository? There is a short tutorial using Zenodo here. JOSS prefers that the name of the archive = the name of the paper, and that the archive author list = paper author list.

The service should give you a DOI for the archive - can you post that here when done? Then I will invoke the ancient spell summoning the Associate Editor in Chief.

tresoldi commented 2 years ago

@majensen I released a version 1.0 (no changes to any file) on GitHub, published on PyPI, and deposited on Zenodo ( https://zenodo.org/record/5552605 ), linking to GitHub and orcid.

The Zenodo DOI is 10.5281/zenodo.5552605

I can of course release a 1.0.1 if necessary, absolutely no problem.

openjournals / joss-reviews