[REVIEW]: Jabberwocky: an ontology-aware toolkit for manipulating text

whedon commented 4 years ago

Submitting author: @sap218 (Samantha Pendleton) Repository: https://github.com/sap218/jabberwocky Version: v1.0.0.0 Editor: @majensen Reviewer: @wdduncan, @balhoff Archive: 10.5281/zenodo.3922261

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/10502055165be490feaa389d51fe99d6"><img src="https://joss.theoj.org/papers/10502055165be490feaa389d51fe99d6/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/10502055165be490feaa389d51fe99d6/status.svg)](https://joss.theoj.org/papers/10502055165be490feaa389d51fe99d6)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@wdduncan & @balhoff, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @majensen know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @wdduncan

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@sap218) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @balhoff

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@sap218) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

majensen commented 4 years ago

@wdduncan thanks - The tutorial can be used to verify functionality manually, so I think the letter of the law is satisfied. That said, @sap218 would you be willing to consider adding a few tests using unittest or pytest? It is an important skill to have, if you haven't done it before, and it makes future support so much easier.

sap218 commented 4 years ago

@wdduncan The manuscript still reflects the software

@wdduncan @majensen

thank you all for the comments throughout and continued patience. I do understand the value of unit testing and plan to explore this aspect of software development when improving the package in the near future, but I am currently constrained by time because of my continued PhD work, which makes it difficult for me to develop this testing skill and update the package - but I hope everyone would agree that the testing repository (and scenario.md) provided does allow users to check the package works as intended (as we have done so for this review) and has sufficient steps describing the functionality of the software.

If the reviewers believe that adding unit tests on top of the system test provided by the test repository should be a pre-requisite of approving the manuscript then I will do my best to invest the time.

majensen commented 4 years ago

@sap218 I understand and I will not hold up a recommendation on this basis.

sap218 commented 4 years ago

@majensen Thank you! I appreciate it :-)

balhoff commented 4 years ago

@majensen @sap218 I agree that the test/example repository is a fine functionality test; so I checked the tests box. I checked the last box, so I approve. Thank you both for your patience!

majensen commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

majensen commented 4 years ago

Hey @sap218 - I added tests to jabberwocky; see PR#13. These basically run the steps in the SCENARIO automatically, using the files in jabberwocky-tests. It's pretty basic, but now you have a working skeleton for more tests in the future. To run the tests, pip install pytest and then run pytest in the repo dir.

sap218 commented 4 years ago

@majensen Thank you so much! I just merged the PR - I appreciate the help!

@balhoff Thank you!

@wdduncan There is now automated testing

majensen commented 4 years ago

@wdduncan there are two boxes unchecked on your review . I think we've taken care of "Automated Tests". The last one is "State of the Field". If you're happy with where we are on these, can you confirm by checking those off. Then I can push our recommendation to the editors-in-chief.

wdduncan commented 4 years ago

@majensen they're checked now :)

majensen commented 4 years ago

@sap218 we're on the final approach. Can you create a tag for your master branch, and then use that to create a public archive for your software? You can use Zenodo -- there's a quick tutorial at https://github.com/openjournals/joss-reviews/issues/1839#issuecomment-560048729 That process will give you a DOI (like "10.5281/zenodo.3558815" ) - please report that back here. Thanks!

sap218 commented 4 years ago

@majensen Done!

10.5281/zenodo.3922261

majensen commented 4 years ago

Excellent @sap218. Now, JOSS prefers that the name of the archive be the same as that of the paper, and that the author lists match. So.. would you be able to update the name and remove that parasitic author from the list on the archive?

sap218 commented 4 years ago

@majensen I believe i have fixed it!

majensen commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

majensen commented 4 years ago

@whedon set 10.5281/zenodo.3922261 as archive

whedon commented 4 years ago

OK. 10.5281/zenodo.3922261 is the archive.

majensen commented 4 years ago

@whedon set v1.0.0.0 as version

whedon commented 4 years ago

OK. v1.0.0.0 is the version.

majensen commented 4 years ago

@whedon accept

whedon commented 4 years ago

Attempting dry run of processing paper acceptance...

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.1093/database/bau033 is OK
- 10.3115/v1/p14-5010 is OK
- 10.1093/nar/gkr972 is OK
- 10.1016/j.ajhg.2008.09.017 is OK
- 10.1093/bib/bbv011 is OK

MISSING DOIs

- https://doi.org/10.25080/majora-92bf1922-00a may be missing for title: Data structures for statistical computing in python

INVALID DOIs

- None

whedon commented 4 years ago

:wave: @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/1523

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/1523, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

majensen commented 4 years ago

Hi @sap218 - can you check on the reference that whedon found missing in https://github.com/openjournals/joss-reviews/issues/2168#issuecomment-651877564 ?

sap218 commented 4 years ago

@majensen just fixed! :-)

danielskatz commented 4 years ago

@majensen - please let me know when this is really ready to go

majensen commented 4 years ago

sure @danielskatz , although you might tell @whedon (or @arfon) not to summon the eic until it performs a dry run free of errors. Sorry to disturb.

sap218 commented 4 years ago

@whedon check references

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.1093/database/bau033 is OK
- 10.3115/v1/p14-5010 is OK
- 10.1093/nar/gkr972 is OK
- 10.1016/j.ajhg.2008.09.017 is OK
- 10.1093/bib/bbv011 is OK
- 10.25080/majora-92bf1922-00a is OK

MISSING DOIs

- None

INVALID DOIs

- None

majensen commented 4 years ago

@openjournals/joss-eics really ready, thanks ever so much.

danielskatz commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

danielskatz commented 4 years ago

👋 @sap218 - I have some comments/requests on the paper, in order from start to end:

"gaining the key textual elements" - I don't know what this means; how does one gain textual elements? Could "gaining" be "determining"?
"having knowledge of the synonyms for clinical terminology may increase the size of the corpus available for analysis" - how is this the case? How does having synonyms increase the size of the corpus?
"e.g. social" <- "e.g., social"
"their chosen ontology and" <- "their chosen ontology, and"
"(Richardson, 2007)) and then" <- "(Richardson, 2007)), and then"
"(e.g. lines from a corpus)" <- "(e.g., lines from a corpus)"
"rank the important terms: using the" <- "rank the important terms by using the"
"command arise will" <- "command, arise, will"
"In Jabberwocky’s test repository (see Jabberwocky repo for further instructions), I show examples of each command separately. In the ‘process’ directory, I combine all three commands to demonstrate an example workflow. With 24 blog posts, the first use of catch returned 11 posts with the provided keywords. With bite I reviewed the CSV of ranked terms and curated new synonyms, simply by adding the corresponding class label from the ontology. I then used arise to add the identified synonyms into the ontology." <- "Jabberwocky’s test repository (see Jabberwocky repo for further instructions) shows examples of each command separately. The ‘process’ directory shows an example that combines all three commands to demonstrate an example workflow. With 24 blog posts, the first use of catch returned 11 posts with the provided keywords. The example uses bite to review the CSV of ranked terms and curated new synonyms, simply by adding the corresponding class label from the ontology. It then uses arise to add the identified synonyms into the ontology."
For the reference Honnibal & Montani, "To appear" is not a journal. Please change this to include the journal name.
For Manning et al., please protect cases with {}s in the bibtex so that the proceedings title appears in Title Case, rather than Sentence case. Also protect the {B} in the title so that it appears as Bloom, not bloom. Also Spacy should be spaCy, I think.
In McKinney et al., do the same. Also protect the {P} in the title so that it appears as Python, not python. Also remove pdfs.semanticscholar.org
In Pedregosa et al., fix Python in the title from python
For Richardson, please add a URL or venue

When these are done, regenerate the pdf with @whedon generate pdf and let me know.

sap218 commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

sap218 commented 4 years ago

@whedon check references

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.1093/database/bau033 is OK
- 10.3115/v1/p14-5010 is OK
- 10.1093/nar/gkr972 is OK
- 10.1016/j.ajhg.2008.09.017 is OK
- 10.1093/bib/bbv011 is OK
- 10.25080/majora-92bf1922-00a is OK

MISSING DOIs

- None

INVALID DOIs

- None

sap218 commented 4 years ago

@danielskatz I have updated the paper and addressed your comments

danielskatz commented 4 years ago

Thanks - I think one was missed, however: the proceedings title for Manning et al. should be in Title Case, not Sentence case.

sap218 commented 4 years ago

@danielskatz Ah yes sorry! Just fixed that now :-)

danielskatz commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

danielskatz commented 4 years ago

@whedon accept

whedon commented 4 years ago

Attempting dry run of processing paper acceptance...

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.1093/database/bau033 is OK
- 10.3115/v1/p14-5010 is OK
- 10.1093/nar/gkr972 is OK
- 10.1016/j.ajhg.2008.09.017 is OK
- 10.1093/bib/bbv011 is OK
- 10.25080/majora-92bf1922-00a is OK

MISSING DOIs

- None

INVALID DOIs

- None

whedon commented 4 years ago

:wave: @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/1526

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/1526, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

danielskatz commented 4 years ago

@whedon accept deposit=true

whedon commented 4 years ago

Doing it live! Attempting automated processing of paper acceptance...

openjournals / joss-reviews