search

openjournals / jose-reviews

Reviews for the Journal of Open Source Education (JOSE)
http://jose.theoj.org
Creative Commons Zero v1.0 Universal
33 stars 4 forks source link

[REVIEW]: EPIDEMIC: Epidemiology Educational Code #149

Closed whedon closed 2 years ago

whedon commented 2 years ago

Submitting author: @BrunaPavlack (Bruna Silveira Pavlack) Repository: https://github.com/americocunhajr/EPIDEMIC Version: v2.0.0 Editor: @magsol Reviewer: @dalonsoa, @3mmaRand Archive: 10.5281/zenodo.5930686

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

status

Status badge code:

HTML: <a href="https://jose.theoj.org/papers/0175828600d6a50dc617d20f9505ef28"><img src="https://jose.theoj.org/papers/0175828600d6a50dc617d20f9505ef28/status.svg"></a>
Markdown: [![status](https://jose.theoj.org/papers/0175828600d6a50dc617d20f9505ef28/status.svg)](https://jose.theoj.org/papers/0175828600d6a50dc617d20f9505ef28)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@dalonsoa & @3mmaRand, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/jose-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @magsol know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @dalonsoa

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @3mmaRand

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

whedon commented 2 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @dalonsoa, @3mmaRand it looks like you're currently assigned to review this paper :tada:.

:warning: JOSE reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSE is currently operating in a "reduced service mode".

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/jose-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/jose-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
whedon commented 2 years ago

Wordcount for paper.md is 2087

whedon commented 2 years ago 
Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.05 s (507.9 files/s, 103749.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
MATLAB                          22            672           1807           2547
Markdown                         2             74              0            304
TeX                              1              8              0             78
YAML                             2              1              4             20
-------------------------------------------------------------------------------
SUM:                            27            755           1811           2949
-------------------------------------------------------------------------------

Statistical information for the repository '0a5c494b616946c1c97e42b7' was
gathered on 2021/11/11.
No commited files with the specified extensions were found.
whedon commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

whedon commented 2 years ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- None

MISSING DOIs

- 10.1101/2020.04.13.20063354 may be a valid DOI for title: SimCOVID: Open-Source Simulation Programs for the COVID-19 Outbreak
- 10.1016/j.amc.2018.06.024 may be a valid DOI for title: Calibration of a SEIR–SEI epidemic model to describe the Zika virus outbreak in Brazil
- 10.1186/1471-2105-13-76 may be a valid DOI for title: EpiFire: An open source C++ library and application for contact network epidemiology
- 10.1109/mis.2013.114 may be a valid DOI for title: Recent Advances in Computational Epidemiology
- 10.1063/5.0005204 may be a valid DOI for title: Embedded model discrepancy: a case study of Zika modeling
- 10.4172/2155-6113.1000e107 may be a valid DOI for title: The Role of Computational Epidemiology and Risk Analysis in the Fight Against HIV/AIDS

INVALID DOIs

- None
3mmaRand commented 2 years ago

I can find no installation instructions in the Readme, the paper or the tutorial

BrunaPavlack commented 2 years ago

I can find no installation instructions in the Readme, the paper or the tutorial

Hi! First of all, thank you so much for agreeing to review our code @3mmaRand! The code runs in MATLAB or GNU Octave software. Put the MATLAB working directory where the code is and run via terminal. Let's add in the materials how to install and use. Thank you!

BrunaPavlack commented 2 years ago

Hi! We added a part about installation and use in the tutorial and in the read me. Thanks for the comment!

whedon commented 2 years ago

:wave: @3mmaRand, please update us on how your review is going (this is an automated reminder).

whedon commented 2 years ago

:wave: @dalonsoa, please update us on how your review is going (this is an automated reminder).

dalonsoa commented 2 years ago

@magsol, for some reason, I am unable to tick the checkboxes above as my review progresses. Could you check what is going on?

magsol commented 2 years ago

@dalonsoa I am unsure. @labarba, any reason a reviewer added to the issue would be unable to tick the checkboxes?

magsol commented 2 years ago

@3mmaRand 👋 Hi! How is the review going? Anything I can help with? Any update since the addition of installation instructions?

labarba commented 2 years ago

It is likely that the reviewer did not accept the invitation from GitHub, which sets the required permissions.

labarba commented 2 years ago

@whedon re-invite @dalonsoa as reviewer

whedon commented 2 years ago

The reviewer already has a pending invite.

@dalonsoa please accept the invite by clicking this link: https://github.com/openjournals/jose-reviews/invitations

magsol commented 2 years ago

Ahhhh. There it is. @dalonsoa please be sure to accept the GitHub invite and let me know if that fixes things.

dalonsoa commented 2 years ago

I've just tried. It says it has expired :(

magsol commented 2 years ago

@whedon re-invite @dalonsoa as reviewer

whedon commented 2 years ago

OK, the reviewer has been re-invited.

@dalonsoa please accept the invite by clicking this link: https://github.com/openjournals/jose-reviews/invitations

magsol commented 2 years ago

@dalonsoa Give it another go?

dalonsoa commented 2 years ago

Fantastic! It works. Many thanks!

3mmaRand commented 2 years ago

hello @magsol @BrunaPavlack The paper describes the software as aimed at " ..... students who are interested in computational epidemiology. The programs and tutorials are designed to offer good introductory material." I have lots of biology students who would be interested in this! But they wouldn't be able to use it because they would need more instruction for using matlab/octave. So maybe describe the software as aimed at matlab users intersted in computational epidemiology or add a bit more instruction>

BrunaPavlack commented 2 years ago

hello @magsol @BrunaPavlack The paper describes the software as aimed at " ..... students who are interested in computational epidemiology. The programs and tutorials are designed to offer good introductory material." I have lots of biology students who would be interested in this! But they wouldn't be able to use it because they would need more instruction for using matlab/octave. So maybe describe the software as aimed at matlab users intersted in computational epidemiology or add a bit more instruction>

Hello @3mmaRand ! Thank you very much for your observation! We have already updated the paper the fact that it is interesting that the user has previous knowledge of Octave or MATLAB. On the other hand, as the organization of the EPIDEMIC code is done in a very didactic way, those students who have first access to these programming languages ​​may also be able to reproduce the simulations. Little by little, we are always updating the tutorial and adding more information about the use of GNU Octave and MATLAB software.

dalonsoa commented 2 years ago

I have a few comments, mostly focussed on the code and the documentation. I have not checked the paper itself, yet:

Version

There are no tags or releases in the GitHub repo matching the version indicated up in this thread (1.0.0). They should be created.

Although not required by JOSE, I would recomend getting a DOI for the tool via Zenodo or Figshare, for example, to help citing and reproducibility.

Installation instructions

The documentation file says in page 9:

After downloading, just run the script. The working directory of compatible software is placed in the place where the EPIDEMIC code is and it is executed via the terminal.

This is not helpful. What script should be run? From where? What "directory of compatible software" means? The next page shows a screenshot, but still it does not provide proper instructions on how to use the tool, specially for those unfamiliar with how Octave or Matlab operate.

There should also be an statement about what version(s) of Matlab or Octave are required to run the tool or, at the very least, in which versions it has been run. If extra packages or toolkits are required, it also needs to be mentioned.

Documentation content and CONTRIBUTING guidelines

Keeping aside improving the installation/usage instructions, I miss documenting the API and how the code itself is structured, so potential contributiors know where to find the different components and improve them or add new features. In this respect, there is no CONTRIBUTING guidelines, neither. There is a message at the very end of the README indicating to send an email, but to me that is inssuficient for an open source software.

Testing

There are no test, manual or automated, to run and prove that things work as expected. Octave has an easy way of including tests, so I strongly recomend the authors make use of it throughout the code and either setup an automated testing workflow using GitHub Actions or provide instructions on how to run the tests.

Documentation format

Finally, having the documentation only as a monolythinc PDF file is pretty hard to use. It is not easy to navigate to the section of interest, even though it has a ToC that you can use to jump to one place to another. Changing that to a more accesible format (online, in GitHub pages which is already setup, or Read the Docs, for example) will be really useful - although I understand it is a lot of work. Not necessary now, but really something to consider in the future.

BrunaPavlack commented 2 years ago

Hello @dalonsoa ! Thank you so much for the comments! We are making the adjustments and so to finish we communicate here.

magsol commented 2 years ago

Hey folks! Any progress here on implementing the reviewer's feedback?

BrunaPavlack commented 2 years ago

Hey folks! Any progress here on implementing the reviewer's feedback?

Hello! sorry for the delay in getting back. We are finalizing implementations of feedback received by reviewers. We have already updated some suggestions in the tutorial. We've added more installation instructions and also added in readme more information about contributions. We need to finish the question of updating the tests and also the version to put in ZENODO. About the format of the tutorial, we are already discussing to update according to the suggestions, however we will do more in the future. Any new information will be posted here. Thanks!

BrunaPavlack commented 2 years ago

Hi! We put the code in the Zenodo. Can be found via the link https://doi.org/10.5281/zenodo.5930686

TosinMA commented 2 years ago

Hello @dalonsoa ! About the tests:

The authors create two ways to verify the usability of the code. The first was the publication of the full code in the Code Ocean platform, which allows anyone to run all the routines e clearly see the response log and figures. It can be accessed by the url https://codeocean.com/capsule/8871213/tree . Moreover, there is an error script to check each function type routine (which are the RHS model functions). These are localized inside the respective subfolders. The authors are available to further clarification if necessary.

TosinMA commented 2 years ago

Hello @dalonsoa ! there was a mistake about the Code Ocean URL. The correct one is https://codeocean.com/capsule/7566849/tree/v1 .

Sorry about this.

BrunaPavlack commented 2 years ago

Hello! Any feedback on the fixes we've made?

magsol commented 2 years ago

Hi @3mmaRand @dalonsoa:

Would you be able to provide any feedback on the fixes the authors have made? Are the fixes sufficient for publication, or are there more changes needed first?

dalonsoa commented 2 years ago

Hi all. Apologies for the slow reponse.

I've re-reviwed the repositorya and changes. It looks really good - definetly very dydactic; I've learnt a lot about epidemiology reading thorught the documentation! I've ticked most of the boxes above. From the end-user perspective, this is ready to go.

Now, I still think that, from a developers and sustainability perspective, this needs a bit more work:

I hope this helps!

BrunaPavlack commented 2 years ago

Thank you so much for the feedback @dalonsoa ! We are adding suggestions. We have already created a document contributing and are improving it. About the API, if you can give us more suggestions on how we can do it. Thank you very much in advance!

dalonsoa commented 2 years ago

I'm used to Python, where documenting the API is pretty easy in several ways. It seems that for Octave, this is less straigth forward. Have a look at this comment in StackOverflow (and links therein) for a suggestion on how to do it:

https://stackoverflow.com/a/42030298/3778792

Please, give it a try and check how complex this approach is to use. It might take a while to sort that put, so I'm happy to tick this off as long as there is a clear commitment and roadmap from your side to sort that out (i.e. open an issue in the repo to tackle this).

3mmaRand commented 2 years ago

Aplogies from me too! Once term starts, I'm pushed for time. I have now viewed the changes and agree that the repo and paper are ready for the end user. Nice work, well done @BrunaPavlack!

BrunaPavlack commented 2 years ago

@dalonsoa and @3mmaRand thank you so much for all the contributions! We added a document about the contribution and as the code is used we improve. It was also added to the tutorials about test routines. Regarding the code documentation, we made a file called CODE DOCUMENTATION.md where we explained the functions used in the compartmental models and the inputs and outputs of the codes of the modules Modeling, Trends and Forecasts. We are currently updating this document in line with code improvements. If the user needs information about the functions in Octave, it is possible to put, for example, help rhs_SIR and information about the function will appear on the screen. This information will also be added in the tutorials.

BrunaPavlack commented 2 years ago

Hello! Any feedback for us to add?

dalonsoa commented 2 years ago

HI @BrunaPavlack , I'm really sorry. I seem to have missed the notification for your previos message.

I am happy to accept the repo as it is now. Well done!

magsol commented 2 years ago

Fantastic, thank you so much @dalonsoa and @3mmaRand!

magsol commented 2 years ago

@whedon generate pdf

whedon commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

magsol commented 2 years ago

@whedon check references

whedon commented 2 years ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- None

MISSING DOIs

- 10.1101/2020.04.13.20063354 may be a valid DOI for title: SimCOVID: Open-Source Simulation Programs for the COVID-19 Outbreak
- 10.1016/j.amc.2018.06.024 may be a valid DOI for title: Calibration of a SEIR–SEI epidemic model to describe the Zika virus outbreak in Brazil
- 10.1186/1471-2105-13-76 may be a valid DOI for title: EpiFire: An open source C++ library and application for contact network epidemiology
- 10.1109/mis.2013.114 may be a valid DOI for title: Recent Advances in Computational Epidemiology
- 10.1063/5.0005204 may be a valid DOI for title: Embedded model discrepancy: a case study of Zika modeling
- 10.4172/2155-6113.1000e107 may be a valid DOI for title: The Role of Computational Epidemiology and Risk Analysis in the Fight Against HIV/AIDS

INVALID DOIs

- None
magsol commented 2 years ago

@BrunaPavlack This looks great!!! Last bit to get this pushed out:

Could you create a tagged release and archive, and report the version number and archive DOI? I'll take it from there!

BrunaPavlack commented 2 years ago

Hi! Thank you so much for all your contribution @magsol ! The tag was already created with the version. I updated by putting the DOI. Please confirm that this is correct!

magsol commented 2 years ago

@whedon set 10.5281/zenodo.5930686 as archive

whedon commented 2 years ago

OK. 10.5281/zenodo.5930686 is the archive.

magsol commented 2 years ago

@whedon set v1.0.0 as version