search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
721 stars 38 forks source link

[REVIEW]: PyVOLCANS: A Python package to flexibly explore similarities and differences between volcanic systems #3649

Closed whedon closed 2 years ago

whedon commented 3 years ago

Submitting author: @PTierz (Pablo Tierz Lopez) Repository: https://github.com/BritishGeologicalSurvey/pyvolcans Version: v1.3.2 Editor: @elbeejay Reviewer: @jifarquharson, @meghanrjones Archive: 10.5281/zenodo.5809105

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/5148a124aa6c9aecf5936022dd1a2bf3"><img src="https://joss.theoj.org/papers/5148a124aa6c9aecf5936022dd1a2bf3/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/5148a124aa6c9aecf5936022dd1a2bf3/status.svg)](https://joss.theoj.org/papers/5148a124aa6c9aecf5936022dd1a2bf3)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@jifarquharson & @meghanrjones, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @elbeejay know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @jifarquharson

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @meghanrjones

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @jifarquharson, @meghanrjones it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
whedon commented 3 years ago

PDF failed to compile for issue #3649 with the following error: 

 Can't find any papers to compile :-(
whedon commented 3 years ago 
Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.08 s (129.7 files/s, 48939.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                           8            640           1605           1714
Markdown                         1             35              0             97
YAML                             2              9              0             52
-------------------------------------------------------------------------------
SUM:                            11            684           1605           1863
-------------------------------------------------------------------------------

Statistical information for the repository '943343d3818aa42e4d6b7546' was
gathered on 2021/08/23.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
John A Stevenson                17          2649            186           41.44
PTierz                         159          1897            729           38.39
Tierz Lopez                     10           192             54            3.60
mobiuscreek                     22           405            276            9.95
ptierz                           1             1              1            0.03
vyronc                          30           251            200            6.59

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
John A Stevenson           2560           96.6          7.5               15.27
PTierz                     1158           61.0          3.3               11.23
mobiuscreek                  43           10.6          0.6                0.00
vyronc                      198           78.9          5.7                1.52
elbeejay commented 3 years ago

@whedon generate pdf from branch JOSS_paper

whedon commented 3 years ago 
Attempting PDF compilation from custom branch JOSS_paper. Reticulating splines etc...
whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

elbeejay commented 3 years ago

Hi @jifarquharson and @meghanrjones, thank you both again for agreeing to review this submission! We are currently asking reviewers to try and complete their reviews in 6 weeks. whedon will automatically send you a reminder after 2 weeks, and I'll be checking in throughout the process to answer any questions. If you find you need more time to review just say so in the thread here.

The JOSS review process is entirely open and transparent, and takes place on GitHub. Review comments should be made as issues in the PyVOLCANS repository, please just link this review issue when doing so to help us keep track of things (paste https://github.com/openjournals/joss-reviews/issues/3649 into the issue text).

For reference, links to the JOSS reviewer criteria and checklist are below as well:

Feel free to ping me (@elbeejay) if you have any questions or concerns. Thanks again for agreeing to review for JOSS.

maxrjones commented 3 years ago

Great, @elbeejay can you please assign this issue to me as a reviewer? Or can I request whedon actions for this?

elbeejay commented 3 years ago

@meghanrjones I can try to have whedon re-send the invite, but I believe you should have received an email when this issue was originally opened.

elbeejay commented 3 years ago

@whedon re-invite @meghanrjones as reviewer

whedon commented 3 years ago

@meghanrjones already has access.

whedon commented 3 years ago

:wave: @jifarquharson, please update us on how your review is going (this is an automated reminder).

whedon commented 3 years ago

:wave: @meghanrjones, please update us on how your review is going (this is an automated reminder).

elbeejay commented 3 years ago

Hello all, looks like @jifarquharson has been opening up some issues and has finished (or nearly finished) their review of this submission. @meghanrjones I wanted to check in to see how things are looking on your end. When do you anticipate being able to review PyVOLCANS?

Thanks all and thanks to @PTierz and co-authors for both their submission and the revisions they have made thus far in response to review comments.

jifarquharson commented 3 years ago

Hi @elbeejay,

I have read the article, "PyVOLCANS: A Python package to flexibly explore similarities and differences between volcanic systems " with great interest. I have spent a good deal of time using the PyVOLCANS package in the course of this review, and I am really happy with the functionality. For the most part, the documentation was sufficient to ensure smooth operation of the tool, with the exception of this issue which was resolved swiftly. All items in the Review checklist have been checked off.

The article itself is concise and well written, and ably demonstrates the community requirement for this open-source tool. Just a note: "Eitnat Lev" is thanked in the acknowledgements, which is presumably a typo.

The major outstanding component, as discussed in the pre-review stage, is the conversion of the original analogy matrices to Python. However, I would contend that this should not be a barrier for publication at this time, although I believe it should be a longer-term goal of the PyVOLCANS team. Given that the methodology is well documented elsewhere (Tierz et al. 2019), I am optimistic that contributions from the wider community will facilitate its wholesale conversion into an open-source format.

PyVOLCANS is a timely and useful tool, with immediate and obvious utility not just for geoscience researchers, but also hazard analysts, policy developers, civil protection authorities, and other invested parties. Once issue #4 is resolved, I'd be happy to recommend publication.

maxrjones commented 3 years ago

Hello all, looks like @jifarquharson has been opening up some issues and has finished (or nearly finished) their review of this submission. @meghanrjones I wanted to check in to see how things are looking on your end. When do you anticipate being able to review PyVOLCANS?

Thanks all and thanks to @PTierz and co-authors for both their submission and the revisions they have made thus far in response to review comments.

Hi @elbeejay, thanks for checking in. I'll provide a review within the next week.

PTierz commented 3 years ago

Dear @elbeejay, @jifarquharson and @meghanrjones,

Thank you very much for your messages and for all your work on the review of our manuscript.

@jifarquharson - We are hugely thankful for your very positive comments on PyVOLCANS and its potential applications. Regarding the typo, many thanks for pointing it out. I have corrected it in the paper.md file, in the JOSS_paper branch (changes are highlighted in bold text). Finally, concerning Issue #4, we hope we will have closed it by the end of this week. Please do not hesitate to let us know if you require anything else from us.

Very many thanks all once again.

Best regards, Pablo (on behalf of the co-authors)

elbeejay commented 3 years ago

Hello all, wanted to check-in and see how the review process is going.

It appears as if the issue raised by @meghanrjones (https://github.com/BritishGeologicalSurvey/pyvolcans/issues/14) has been addressed - if this is the case can the issue be closed? @meghanrjones at this time I'll also ask you to either raise concerns about your outstanding unchecked boxes in the checklist above, or to check those off and certify that you are happy with the current state of the paper and software.

Thanks all, Jay

maxrjones commented 3 years ago

Dear all,

I read the article "PyVOLCANS: A Python package to flexibly explore similarities and differences between volcanic systems” and reviewed the software with great interest. The article describes a novel, open-source tool for accessing the results from the VOLcano ANalogues Search (VOLCANS). PyVOLCANS identifies candidate analog volcanoes based on a combination of volcanological criteria, with options to select a specific weighting of criteria, provide a priori analog volcanoes for comparison, and generate a summary figure from the results.

The article is well-written and clearly articulates the benefit of PyVOLCANS for volcanic hazard assessment. PyVOLCANS holds great potential as an open source tool that enables structured and reproducible selection of analog volcanoes.

Below are comments about specific checklist items:

Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?

Coverage reports from pytest-cov suggests that while there are tests covering most of the source code in pyvolcans/pyvolcans_func.py, there is no coverage for the main API in pyvolcans/pyvolcans.py. Since the external documentation covers the CLI and not the python functions, I do not find that there are automated tests or manual steps described for verifying the functionality of the software.

Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines?

My primary concern with this checkpoint is whether the software can be considered feature complete, which is listed as necessary in the eligibility described in the ’Substantial scholarly effort’ section of the JOSS guidelines. Currently, the software reports the analogy scores for the top 10 volcanos for the given target volcano. I do not expect this information alone to be sufficient for conducting scholarly research or teaching exercises. In order for the information to be sufficient, I expect that the user would need to be able to query the numerical and categorical variables for both the target and the analog volcanoes (https://github.com/BritishGeologicalSurvey/pyvolcans/issues/16).

In the article and the pre-review discussion (e.g., https://github.com/openjournals/joss-reviews/issues/3436#issuecomment-885611976), the authors cite four scientific studies that use the results from the VOLCANS study as evidence for the obvious research applications and usefulness of PyVOLCANS (White, 2020; Simmons, 2020; Crummy et al., 2021, and Tierz et al., 2020). I was unable to evaluate the relevance of PyVOLCANS relative to VOLCANS for three of these citations (White, 2020; Simmons, 2020, and Crummy et al., 2021), since there were no DOI’s provided for these theses/internal reports. The final cited paper (Tierz et al., 2020) would not have been able to use PyVOLCANS in its current form because they needed to remove the rock type “Dacite” from the GVP profile of Aluto volcano, which would require recomputing the analogy matrices, and no documentation is provided for how to use alternative analogy matrices. Are there pre-prints available for the non-peer-reviewed examples that demonstrate that PyVOLCANS could have been used for this research? Are other examples available that demonstrate the obvious research application of PyVOLCANS in its current form?

Other features that I would have found useful but do not consider essential for considering the package ‘feature complete’ include:

Other comments

One small comment regarding the title/summary: since there is no API documentation for the Python functions, the fact that users interact with PyVOLCANS via the CLI is essential information for those interested in using the tool. Therefore, this information should be included in the title and/or summary.

volcan01010 commented 3 years ago

Hi @meghanrjones and @jifarquharson,

Thanks for taking the time to give such thorough reviews of PyVOLCANS. This is the first time that I've submitted anything to JOSS and it is very helpful to get constructive feedback from people that have really engaged with the software.

I agree with @meghanrjones's main comments about testing and extra features.

Test coverage: we can certainly increase the test coverage and include the CLI component. For now, only some of the trickiest functions have tests.

Feature completion: we also think that adding the ability to see the data for each volcano would be a valuable addition. Much of the work to do this would also contribute to the porting of the Matlab parts of the code to Python. We had discussed adding this as part of a v2 release as it would involve significant extra code, along with refactoring existing code.

In general, we have tried to take a "release early, release often" approach to pyVOLCANS. We published the package as soon as we had something that was genuinely useful e.g. users can interrogate the pyVOLCANS matrices and find the best analogues for a chosen volcano. This way scientists can benefit immediately, without having to wait for the full porting of the Matlab code. This is the main feature, with the volcano source data and porting of the Matlab code being extensions. Since then we have incrementally improved functionality through subsequent releases.

We agree that showing metadata and porting the Matlab code would be valuable additions. However, the funding that we used for the main development has ended and we are now working sporadically where we can between other commitments. It could take a few weeks to add the volcano data and months to replace the full Matlab code. How long can a JOSS paper remain in review for, @elbeejay?

elbeejay commented 3 years ago

How long can a JOSS paper remain in review for, @elbeejay?

JOSS does not have a restriction on how long a paper can remain in review. In the event larger changes are going to be made, or authors need a period of weeks or months to address reviewer comments, we simply ask for an estimate from the authors of when revisions are expected to be completed. This is so that reviewers know when to anticipate being asked to have another look at the newly revised package. We can also set reminders using whedon so that editors and reviewers will be reminded to check back on the review issue after an agreed upon period of time.

So all of that being said, and given @meghanrjones thoughtful review, I would suggest the authors develop a plan to address the comments and give us an estimate of when the changes will be completed.

Thanks all

PTierz commented 3 years ago

Dear @meghanrjones and @elbeejay,

Thank you very much for your reviews and comments on our submitted manuscript. They all contribute to improving our software tool and, hence, they are greatly appreciated.

As a summary for all the requests, we sincerely think that solving Issue #16 is timely and highly beneficial. This is a feature that we had previously thought about implementing. Many thanks for the prompt to do it now. I consider that we can address the issue within the next 3-4 weeks, approximately.

Concerning some of the specific comments raised by @meghanrjones, please allow us to reply in detail to them (below, the reviewer comments are shown in bold text and our replies in italic text. Many thanks):

My primary concern with this checkpoint is whether the software can be considered feature complete, which is listed as necessary in the eligibility described in the ’Substantial scholarly effort’ section of the JOSS guidelines. Currently, the software reports the analogy scores for the top 10 volcanos for the given target volcano. I do not expect this information alone to be sufficient for conducting scholarly research or teaching exercises. In order for the information to be sufficient, I expect that the user would need to be able to query the numerical and categorical variables for both the target and the analog volcanoes (BritishGeologicalSurvey/pyvolcans#16).

In the article and the pre-review discussion (e.g., #3436 (comment)), the authors cite four scientific studies that use the results from the VOLCANS study as evidence for the obvious research applications and usefulness of PyVOLCANS (White, 2020; Simmons, 2020; Crummy et al., 2021, and Tierz et al., 2020). I was unable to evaluate the relevance of PyVOLCANS relative to VOLCANS for three of these citations (White, 2020; Simmons, 2020, and Crummy et al., 2021), since there were no DOI’s provided for these theses/internal reports. The final cited paper (Tierz et al., 2020) would not have been able to use PyVOLCANS in its current form because they needed to remove the rock type “Dacite” from the GVP profile of Aluto volcano, which would require recomputing the analogy matrices, and no documentation is provided for how to use alternative analogy matrices. Are there pre-prints available for the non-peer-reviewed examples that demonstrate that PyVOLCANS could have been used for this research? Are other examples available that demonstrate the obvious research application of PyVOLCANS in its current form?

_Very many thanks for your comments. First of all, we would like to specify that PyVOLCANS can report any number of top analogue volcanoes, not just the top 10 analogues, by using the parameter --count. This, together with the customary choice of weighting scheme and a priori analogues (for any target volcano), provides the software with great flexibility, which is one of the greatest strengths of the presented code._

We sincerely believe that, in its current form, the software already satisfies the requirements to qualify as a ‘Substantial scholarly effort’. In this respect, the text in the JOSS guidelines specifies: “Your software should be a significant contribution to the available open source software that either enables some new research challenges to be addressed or makes addressing research challenges significantly better (e.g., faster, easier, simpler).”

In this regard, PyVOLCANS represents a substantial scholarly effort because it both "enables some new research challenges to be addressed" (as shown in Tierz et al., 2019, 2020; Simmons, 2021) and "makes addressing research challenges significantly better (e.g., faster, easier, simpler)". As concerns this second point, PyVOLCANS represents the first open-source objective, structured and reproducible method to identify sets of analogue volcanoes from global volcanological databases. As we mention at the end of the Summary of our manuscript: “Such methods [of obtaining data and information from analogue volcanoes] have been used for many years but we have created the first tool to enable a structured and harmonised approach worldwide.”

_We shall stress that, with PyVOLCANS, any user anywhere in the world can generate any number of top analogue volcanoes for any Holocene volcano in the Global Volcanism Program database (v4.6.7), using a fully customary weighting scheme that combines five main volcanological criteria, in a matter of a few seconds. We strongly argue that this stands as a major step forward compared to previous approaches to selection of analogue volcanoes. Many thanks._

We would also like to say that we interpret the sentence “enables some new research challenges to be addressed” as having a ‘forward look’ meaning. That is, the presented software tool (PyVOLCANS) should enable addressing such new research challenges in the future, not necessarily by the time of publication of the manuscript.

We also think that it is important to stress that, until March-April 2021 when PyVOLCANS was first released, only myself was able to conduct data-driven searches for analogue volcanoes using VOLCANS. As a matter of fact (and in addition to the scientific studies cited in our submitted manuscript to JOSS), since December 2019 until April 2021, I was personally contacted by colleagues interested to run VOLCANS for 8 different volcanoes in 6 countries (Poás and Turrialba, in Costa Rica; Okmok and Cleveland, in the USA; Aso, in Japan; Gede-Pangrango, in Indonesia; Copahue, in Chile; and Terceira, in Portugal). The uses for these sets of analogue volcanoes ranged from probabilistic volcanic hazard assessment and general understanding of similarities and differences between volcanic systems to outreach purposes. With the release of PyVOLCANS, all these analogue searches can now be run directly by the interested users. We wholeheartedly believe that all these examples are evidence of the ‘substantial scholarly effort’ that PyVOLCANS represents in its current form. Nevertheless, we honestly believe that adding the improvements associated with Issue #16 is an important step towards making PyVOLCANS feature-complete. Very many thanks for your suggestion.

Concerning the specific comments about the bibliographical references included in the manuscript, we would like to specify that:

Finally, we most sincerely believe that the VOLCANS article itself (Tierz et al., 2019) represents good evidence of the obvious research applications of PyVOLCANS. Many thanks.

We honestly hope that these replies can help clarify some of the aspects that were previously unclear and may contribute to considering our PyVOLCANS manuscript suitable for publication in JOSS. If you had any further requests for us, please do not hesitate to let us know. Many thanks.

We will commit to solve Issue #16 as soon as possible over the next 3-4 weeks. Many thanks again.

Best wishes, Pablo (on behalf of the co-authors)

PTierz commented 3 years ago

Dear @meghanrjones, @jifarquharson and @elbeejay,

I sincerely hope that this message finds you well.

I just wanted to let you know that we have preliminarily addressed Issue #16 in a new branch called volcano-id-profile.

We still need to add a few improvements before we merge the branch into main but please feel free to explore the new functionalities of the code and let us know if they appear to be satisfactory as regards the request in the aforementioned Issue.

We will aim at completing the merge by 15th November 2021.

Very many thanks in advance, Pablo

elbeejay commented 3 years ago

Thanks for the update @PTierz

PTierz commented 2 years ago

Dear @meghanrjones, @elbeejay and @jifarquharson,

I hope this message finds you well.

I would like to kindly ask you to have a look at Merge Request #17 in our PyVOLCANS repository, and let us know whether you consider that the incorporated changes address Issue #16 satisfactorily.

A few changes could still be needed as @volcan01010 and/or @mobiuscreek double-check the merge request, but I think the majority of the work to address the aforementioned Issue has been incorporated into the merge request.

Looking forward to hearing from you regarding these new additions to PyVOLCANS. I sincerely hope that they can help the case of making our manuscript suitable for publication in the Journal of Open Source Software.

Very many thanks for all your work to improve our software.

Best regards, Pablo

elbeejay commented 2 years ago

Hi @PTierz,

Thank you for your efforts to add functionality in this new merge request.

You previously noted that additional tests would be written:

Test coverage: we can certainly increase the test coverage and include the CLI component. For now, only some of the trickiest functions have tests

do you expect this work effort to take place in a separate branch / PR?

And I think a more minor but valid suggestion raised by @meghanrjones was:

One small comment regarding the title/summary: since there is no API documentation for the Python functions, the fact that users interact with PyVOLCANS via the CLI is essential information for those interested in using the tool. Therefore, this information should be included in the title and/or summary.

I agree that at minimum this should be incorporated in the summary section of the paper, if not in the title itself to make it clear to readers/users that they will be interacting with the program via the command-line as opposed to from Python itself.

Thanks, Jay

PTierz commented 2 years ago

Hi @elbeejay,

Very many thanks for your message.

I think we may add the tests for pyvolcans.py in a separate merge request from the one concerning volcano-id-profile branch. Many thanks.

Regarding the specification of the fact that the users interact with PyVOLCANS via the CLI, we had included that information in the “Statement of Need” section of our manuscript, where we describe our package from a general to more detailed perspectives. Users can also quickly see that they will interact with PyVOLCANS via the command line in the README.md file, which they encounter when accessing the PyVOLCANS repository. We sincerely hope that these specifications can be satisfactory enough. We would really like to keep the title of the manuscript as it is now: we do not see any inaccurate message in it and we think it is very important that it remains as simple and attractive as possible. Very many thanks in advance for your understanding.

Please also note that we have now added the necessary acknowledgements to @jifarquharson and @meghanrjones, as reviewers of the manuscript, and to yourself, as Associate Editor, in the latest version of our manuscript (please see commit 0b3e1e7 ). Thank you very much for all your work and support.

Best regards, Pablo (on behalf of the co-authors)

elbeejay commented 2 years ago

Hi @elbeejay,

Very many thanks for your message.

I think we may add the tests for pyvolcans.py in a separate merge request from the one concerning volcano-id-profile branch. Many thanks.

Regarding the specification of the fact that the users interact with PyVOLCANS via the CLI, we had included that information in the “Statement of Need” section of our manuscript, where we describe our package from a general to more detailed perspectives. Users can also quickly see that they will interact with PyVOLCANS via the command line in the README.md file, which they encounter when accessing the PyVOLCANS repository. We sincerely hope that these specifications can be satisfactory enough. We would really like to keep the title of the manuscript as it is now: we do not see any inaccurate message in it and we think it is very important that it remains as simple and attractive as possible. Very many thanks in advance for your understanding.

Please also note that we have now added the necessary acknowledgements to @jifarquharson and @meghanrjones, as reviewers of the manuscript, and to yourself, as Associate Editor, in the latest version of our manuscript (please see commit 0b3e1e7 ). Thank you very much for all your work and support.

Best regards, Pablo (on behalf of the co-authors)

Thanks Pablo.

In the interest of our reviewers time, could you ping us all once PR#17 as well as the anticipated PR regarding the additional tests for pyvolcans.py have been merged? I think it will make more sense for us to all have another look once you and your collaborators are finished making, reviewing, and merging the various changes.

-Jay

PTierz commented 2 years ago

Good morning @elbeejay (@jifarquharson, and @meghanrjones),

Many thanks for your message. Certainly. We will do so as soon as the two merges have been carried out. We expect this to happen by close of play on Friday 26th November.

Hopefully, the changes will be satisfactory for you and the reviewers. I personally think that the code will have significantly improved by the time we have incorporated the two PRs.

Very many thanks again for your suggestions to make PyVOLCANS a better product.

All the best, Pablo

PTierz commented 2 years ago

Dear @meghanrjones, @jifarquharson and @elbeejay,

I sincerely hope this message finds you well.

Please let me thank you once again for all your work, feedback and suggestions on how to improve the PyVOLCANS code and our submission of it to the Journal of Open Source Software.

I am elated to let you know that the latest requested changes have now been incorporated into a new v1.3.0 version of the software. This new version includes two major groups of changes: (1) the added functionality of PyVOLCANS to output the ID profiles (i.e. volcanological data available) for both the target volcano and any set of analogue volcanoes generated (please see merged branch volcano-id-profile; and (2) a comprehensive set of new tests implemented to cover the CLI component in the pyvolcans.py file of the package (please see merged branch test-cli).

You can check the first major addition by running commands like: pyvolcans Kilauea -v (which prints the Top 10 analogues to Kīlauea as well as the ID profile for the target volcano into the stdout); pyvolcans Kilauea -v --output_volcano_data (which saves a .json file containing Kīlauea's ID profile); and pyvolcans Kilauea -v --output_analogues_data (which saves a different .json file containing the ID profile for all the Top 10 analogues to Kīlauea, the latter considering the default, equal-weighting scheme used).

In terms of test coverage, you can now check that pytest --cov=pyvolcans test\ should inform that over 85% and 90% of pyvolcans.py and pyvolcans_func.py, respectively, are now covered by tests included in our PyVOLCANS package.

We most sincerely hope that these additions to our software can significantly help the case of it meriting publication on the Journal of Open Source Software.

Please do not hesitate to let us know if you required anything else from us.

Best regards, Pablo (on behalf of the co-authors)

PTierz commented 2 years ago

Dear @meghanrjones, @jifarquharson and @elbeejay,

I just wanted to let you know that we have also updated the main README file in the PyVOLCANS GitHub repository, as well as the PyVOLCANS Wiki pages to reflect all changes incorporated up to PyVOLCANS v1.3.0. If you had any comments or requests, please let us know.

Looking forward to hearing from you. Thank you very much in advance for your time.

Best regards, Pablo (on behalf of the co-authors)

maxrjones commented 2 years ago

Thanks for addressing our comments. I will take a look at the revisions within the next two weeks and edit the original review checklist as needed.

PTierz commented 2 years ago

Many thanks, @meghanrjones. Please let us know if you required anything else from us during your review, OK? Best regards, Pablo

PTierz commented 2 years ago

Dear @meghanrjones, @jifarquharson and @elbeejay,

I hope this message finds you well.

I just wanted to inform you that I have added two recent, but very relevant, citations to our JOSS manuscript, which further show the future potential and applicability of VOLCANS/PyVOLCANS to the field of volcanic hazard assessment (please see commits 04095ff, ed88cb6).

I considered that it would be important for future users of the code, and hopefully for future readers of our manuscript, to know more about the context and application of the presented software tool.

Thank you very much in advance.

Kind regards, Pablo

elbeejay commented 2 years ago

@whedon generate pdf from branch JOSS_paper

whedon commented 2 years ago 
Attempting PDF compilation from custom branch JOSS_paper. Reticulating splines etc...
whedon commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

elbeejay commented 2 years ago

@PTierz just so you are aware, I will be asking you to merge the JOSS_paper branch into the main branch at the end of this process. I haven't checked if there are any merge conflicts but wanted to mention this.

PTierz commented 2 years ago

Hi @elbeejay,

Very many thanks for your message. Hopefully, it shouldn't be any merge conflicts as we were strictly working on the paper folder.

Would you like us to create the PR for the branch now or after @meghanrjones's reviews?

Thank you very much again.

Best regards, Pablo

elbeejay commented 2 years ago

I will just remind you to merge it in as one of the final steps, there's no need to do so now. Just wanted you to be aware. :+1:

elbeejay commented 2 years ago

Hello @meghanrjones and @jifarquharson,

Wanted to reach out to you both about this review.

@meghanrjones - have you had a chance to look over the recent revision to the repository? No problem if not; please just update us with an anticipated timeline.

@jifarquharson - several revisions to the submission have taken place since your review was conducted. Would you be willing to have a second look through these changes? Please give us an estimate as to when you will be able to do this check-up.

Thank you both, Jay

jifarquharson commented 2 years ago

Hi @elbeejay,

several revisions to the submission have taken place since your review was conducted. Would you be willing to have a second look through these changes? Please give us an estimate as to when you will be able to do this check-up.

I have been keeping up with these as they come in, and all the changes seem to be useful and proportional. I will re-test pyVOLCANS tomorrow and note any additional comments that might be warranted.

Regards, Jamie

maxrjones commented 2 years ago

Hi @elbeejay,

I apologize for the delay - I have not had a chance to finish the review yet. I will complete the review tomorrow (Saturday 12/18).

Regards,

Meghan

jifarquharson commented 2 years ago

Dear @elbeejay and @PTierz et al.,

I have re-read the article, "PyVOLCANS: A Python package to flexibly explore similarities and differences between volcanic systems," and have tested the additional pyVOLCANS functionality (now v1.3.0). In particular, the improved --output_analogues_data and --output_volcano_data options increase the range of applications of the tool. I can foresee immediate use-cases of this updated functionality. That being said, a small issue exists that hinders their full utility (there is no disambiguation of .json filenames, and files can be overwritten without warning).

Pending a fix to #20, I am really happy with the revised version and heartily recommend its publication. The minor amendments to the text are warranted, with the added references further demonstrating the timely nature of this contribution. I'm satisfied that all of the checklist targets have been met.

PTierz commented 2 years ago

Dear @jifarquharson,

Thank you very much for all the time and work that you have dedicated to the review of our PyVOLCANS code and manuscript. It is hugely appreciated.

Likewise, very many thanks for your support and positive comments about the tool. We sincerely hope that it can be used by and useful for as many volcano scientists and enthusiasts as possible.

We have just posted a comment on Issue #20, explaining how we have addressed it and suggesting some acceptance criteria to be checked before the Issue can be closed.

Thank you so much once again.

Best wishes, Pablo (on behalf of the co-authors)

jifarquharson commented 2 years ago

Happy to report that with cca2fd6 and the update to v1.3.1, the disambiguation/overwriting problem (#20) is fixed.

maxrjones commented 2 years ago

Dear all,

I also re-read the article “PyVOLCANS: A Python package to flexibly explore similarities and differences between volcanic systems” and tested out the functionality added since my initial review using version 1.3.1. I am pleased to report that the changes have satisfactorily addressed my concerns for the remaining checklist items "Substantial scholarly effort” and "Automated tests”. I have edited my review checklist accordingly and recommend publication in JOSS after removing the bold formatting on sections that represent changes in the article text. I appreciate the authors' efforts in making the results from VOLCANS accessible for scholarly research through an open-source tool.

I agree with the authors that the current form of the package will be a useful resource for instructors and researchers. While my recommendation for publication does not require additional changes, I advise that the authors consider adding badges to their readme to facilitate quick evaluation of the build/test status, test coverage, and PyPI project page. I also expect that the package would reach a broader audience if the authors add documentation examples for loading the .JSON and .csv output into commonly used Python objects such as a pandas.DataFrame or provide a complementary, simple Python API.

Sincerely,

Meghan Jones

elbeejay commented 2 years ago

Thanks for the quick replies @jifarquharson and @meghanrjones, your comments and review of this work are greatly appreciated.

@PTierz I would like to ask that you finalize the paper.md file at this time, as Meghan pointed out there is bold formatting in sections that represent changes that are likely not intended to be bold in the final version. I would also suggest the authors consider the suggestions made by Meghan in the above post, namely:

consider adding badges to their readme to facilitate quick evaluation of the build/test status, test coverage, and PyPI project page

and

add documentation examples for loading the .JSON and .csv output into commonly used Python objects such as a pandas.DataFrame or provide a complementary, simple Python API

While neither of these things is a requirement, I do agree that they would increase the user-friendliness of the package and may help people newer to Python or PyVOLCANS.

In any case, please notify me when you've finalized the paper so that I can go through and type-check it for any minor formatting or reference problems.

Thanks, Jay

PTierz commented 2 years ago

Dear @meghanrjones and @elbeejay,

Thank you so very much for your messages and for all the work that you have dedicated to reviewing our PyVOLCANS software and manuscript. This has resulted in significant improvements to PyVOLCANS and we are greatly thankful for that.

Many thanks for your additional suggestions too. We will commit to start working on them after the Christmas break.

I have now removed the bold text from the paper.md file (please see commit 97cbb5b).

Please do not hesitate to let me know about any other task that I should complete.

Best wishes, Pablo (on behalf of the co-authors)

elbeejay commented 2 years ago

@whedon generate pdf from branch JOSS_paper