Submission Group 10: candacovidmetrics (Python)

Submitting Author: Name @adammorphy Package Name: CanadaCovidMetrics One-Line Description of Package: Return up to date covid-19 data in Canada in usable Pandas DataFrames Repository Link: https://github.com/UBC-MDS/canadacovidmetrics Version submitted: v1.1.0 Editor: TBD Reviewer 1: Cuthbert Chow @cuthchow, Reviewer 2: Francisco Mejia @RamiroMejia Reviewer 3: Margot Vore @voremargot Reviewer 4: Thomas Siu @thomassiu Archive: TBD Version accepted: TBD

Description

Include a brief paragraph describing what your package does:

CanadaCovidMetrics uses OpenAPI to extract and format up to date covid data in Canada into Pandas DataFrames. Users can get data on covid cases, recoveries, vaccines, and deaths, broken down by given date intervals and/or Province.

Scope

Please indicate which category or categories this package falls under:
[x] Data retrieval
[x] Data extraction
[x] Data munging
[ ] Data deposition
[ ] Reproducibility
[ ] Geospatial
[ ] Education
[ ] Data visualization*

* Please fill out a pre-submission inquiry before submitting a data visualization package. For more info, see notes on categories of our guidebook.

Explain how the and why the package falls under these categories (briefly, 1-2 sentences):

Our package extracts covid data from an open API, and formats the results into Pandas dataframes

- Who is the target audience and what are scientific applications of this package?

Our package is intended to speed up the proccess of communicating covid information in Canada, and therefore is targeted to science communicators, doctors, data scientists, and others who share covid data.

- Are there other Python packages that accomplish the same thing? If so, how does yours differ?

Existing packages use other APIs which do not update the data consistently, and are not targeted for Canadian data (ex. John Hopkins University API). To our knowledge, no other package exists using OpenCovid API in the Python ecosystem

If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted:

Technical checks

For details about the pyOpenSci packaging requirements, see our packaging guide. Confirm each of the following by checking the box. This package:

[x] does not violate the Terms of Service of any service it interacts with.
[x] has an OSI approved license.
[x] contains a README with instructions for installing the development version.
[x] includes documentation with examples for all functions.
[ ] contains a vignette with examples of its essential functions and uses.
[x] has a test suite.
[x] has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
Publication options
[ ] Do you wish to automatically submit to the Journal of Open Source Software? If so:

JOSS Checks

- [ ] The package has an **obvious research application** according to JOSS's definition in their [submission requirements][JossSubmissionRequirements]. Be aware that completing the pyOpenSci review process **does not** guarantee acceptance to JOSS. Be sure to read their submission requirements (linked above) if you are interested in submitting to JOSS. - [ ] The package is not a "minor utility" as defined by JOSS's [submission requirements][JossSubmissionRequirements]: "Minor ‘utility’ packages, including ‘thin’ API clients, are not acceptable." pyOpenSci welcomes these packages under "Data Retrieval", but JOSS has slightly different criteria. - [ ] The package contains a `paper.md` matching [JOSS's requirements][JossPaperRequirements] with a high-level description in the package root or in `inst/`. - [ ] The package is deposited in a long-term repository with the DOI: *Note: Do not submit your package separately to JOSS*

Are you OK with Reviewers Submitting Issues and/or pull requests to your Repo Directly?

This option will allow reviewers to open smaller issues that can then be linked to PR's rather than submitting a more dense text based review. It will also allow you to demonstrate addressing the issue via PR links.

[x] Yes I am OK with reviewers submitting requested changes as issues to my repo. Reviewers will then link to the issues in their submitted review.

Code of conduct

[x] I agree to abide by pyOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted. P.S. *Have feedback/comments about our review process? Leave a comment here
Editor and Review Templates

Editor and review templates can be found here

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

[X] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

[X] A statement of need clearly stating problems the software is designed to solve and its target audience in README
[X] Installation instructions: for the development version of package and any non-standard dependencies in README
[ ] Vignette(s) demonstrating major functionality that runs successfully locally
[X] Function Documentation: for all user-facing functions
[X] Examples for all user-facing functions
[X] Community guidelines including contribution guidelines in the README or CONTRIBUTING.
[X] Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a setup.py file or elsewhere.

Readme requirements The package meets the readme requirements below:

[X] Package has a README.md file in the root directory.

The README should include, from top to bottom:

[X] The package name
[X] Badges for continuous integration and test coverage, a repostatus.org badge, and any other badges. If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.)
[ ] Short description of goals of package, with descriptive links to all vignettes (rendered, i.e. readable, cf the documentation website section) unless the package is small and there’s only one vignette repeating the README.
[X] Installation instructions
[ ] Any additional setup required (authentication tokens, etc)
[X] Brief demonstration usage
[X] Direction to more detailed documentation (e.g. your documentation files or website).
[X] If applicable, how the package compares to other similar packages and/or how it relates to other packages
[X] Citation information

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole. Package structure should follow general community best-practices. In general please consider:

[X] The documentation is easy to find and understand
[X] The need for the package is clear
[X] All functions have documentation and associated examples for use

Functionality

[X] Installation: Installation succeeds as documented.
[X] Functionality: Any functional claims of the software been confirmed.
[X] Performance: Any performance claims of the software been confirmed.
[X] Automated tests: Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
[X] Continuous Integration: Has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
[X] Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines.

For packages co-submitting to JOSS

[ ] The package has an obvious research application according to JOSS's definition in their submission requirements.

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

[ ] A short summary describing the high-level functionality of the software
[ ] Authors: A list of authors with their affiliations
[ ] A statement of need clearly stating problems the software is designed to solve and its target audience.
[ ] References: with DOIs for all those that have one (e.g. papers, datasets, software).

Final approval (post-review)

[ ] The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing:

1.5 hours

Review Comments

Hello Group 10 members,

Congratulations, you did a great job on your package.

Overall, your repo is well organized, everything is easy to find and your package complies with all the requirements asked in the Milestones.

I have the following comments about your package:

I found some inconsistencies in the Usage & Example section of your README file and in the published documentation. The examples for National vaccination completion in 2021 and for Daily new case count by province are not running correctly because of the name of the 'date' column produced by the query.
In the published documentation, the code is running a Module not Found error in the examples section.
The main functionality of the package is the query of data, so, it would be relevant to show the output datasets (a few rows of data). I think it would be great if you can show more examples on how to query data using different arguments like date or location
You can make your functions more robust by adding exception handling for the argument after. If I enter an invalid date E.g "2000-01-01" or "2019-12-31" we should expect an error message to let the user know the date does not make sense or there is no data for those dates.
I tried to retrieve the deaths from "2020-01-01" to "2020-03-01", it seems like there is no data for those dates, this is an expected outcome because it was the start of the pandemic. However, the error that the function shows is a "KeyError", this could be confusing because the date format entered is correct. You can try to add some exception handling for those dates that do not have data.

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

[x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

[x] A statement of need clearly stating problems the software is designed to solve and its target audience in README
[x] Installation instructions: for the development version of package and any non-standard dependencies in README
[ ] Vignette(s) demonstrating major functionality that runs successfully locally
[x] Function Documentation: for all user-facing functions
[x] Examples for all user-facing functions
[x] Community guidelines including contribution guidelines in the README or CONTRIBUTING.
[x] Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a setup.py file or elsewhere.

Readme requirements The package meets the readme requirements below:

[x] Package has a README.md file in the root directory.

The README should include, from top to bottom:

[x] The package name
[x] Badges for continuous integration and test coverage, a repostatus.org badge, and any other badges. If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.)
[x] Short description of goals of package, with descriptive links to all vignettes (rendered, i.e. readable, cf the documentation website section) unless the package is small and there’s only one vignette repeating the README.
[x] Installation instructions
[ ] Any additional setup required (authentication tokens, etc)
[x] Brief demonstration usage
[x] Direction to more detailed documentation (e.g. your documentation files or website).
[x] If applicable, how the package compares to other similar packages and/or how it relates to other packages
[x] Citation information

Usability

[x] The documentation is easy to find and understand
[x] The need for the package is clear
[x] All functions have documentation and associated examples for use

Functionality

[x] Installation: Installation succeeds as documented.
[ ] Functionality: Any functional claims of the software been confirmed.
[ ] Performance: Any performance claims of the software been confirmed.
[x] Automated tests: Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
[x] Continuous Integration: Has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
[x] Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines.

For packages co-submitting to JOSS

[ ] The package has an obvious research application according to JOSS's definition in their submission requirements.

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

[ ] A short summary describing the high-level functionality of the software
[ ] Authors: A list of authors with their affiliations
[ ] A statement of need clearly stating problems the software is designed to solve and its target audience.
[ ] References: with DOIs for all those that have one (e.g. papers, datasets, software).

Final approval (post-review)

[ ] The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing:

1.5 hours

Review Comments

Hi Group 10,

Congratulations and well done on developing the package! All the contents are well organized, and clear examples are provided in the readme which is really great. The 100% code coverage is really an exceptional job!

Here are some of the comments for your considerations to improve. Note that I created a new conda environment with only python 3.9 installed for the test.

It would be great if you include package dependencies in README so that the users can install it along with this package. For example, when I run the installation and the first example, I realized that matplotlib is required.
In the second function, I was not able to run it successfully with error KeyError: 'date_vaccine_completed'. I also found that the test case does not check against this column (line #119) in test_canadacovidmetrics.py
Similarly, in the example of ccm.get_cases(), the code ccm.get_cases().set_index('date_report') throws error of "None of ['date_report'] are in the columns".
It would be great if examples of get_recoveries is added back to readme examples
To make the function more robust, you may consider adding a check that before must be smaller or equal to after, instead of throwing Key error.

Once again, congratulations team on the nice work!

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

[x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

[x] A statement of need clearly stating problems the software is designed to solve and its target audience in README
[x] Installation instructions: for the development version of package and any non-standard dependencies in README
[] Vignette(s) demonstrating major functionality that runs successfully locally
[x] Function Documentation: for all user-facing functions
[x] Examples for all user-facing functions
[ ] Community guidelines including contribution guidelines in the README or CONTRIBUTING.
[ ] Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a setup.py file or elsewhere.

Readme requirements The package meets the readme requirements below:

[x] Package has a README.md file in the root directory.

The README should include, from top to bottom:

[x] The package name
[x] Badges for continuous integration and test coverage, a repostatus.org badge, and any other badges. If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.)
[x] Short description of goals of package, with descriptive links to all vignettes (rendered, i.e. readable, cf the documentation website section) unless the package is small and there’s only one vignette repeating the README.
[x] Installation instructions
[ ] Any additional setup required (authentication tokens, etc)
[x] Brief demonstration usage
[x] Direction to more detailed documentation (e.g. your documentation files or website).
[x] If applicable, how the package compares to other similar packages and/or how it relates to other packages
[x] Citation information

Usability

[x] The documentation is easy to find and understand
[x] The need for the package is clear
[x] All functions have documentation and associated examples for use

Functionality

[x] Installation: Installation succeeds as documented.
[ ] Functionality: Any functional claims of the software been confirmed.
[ ] Performance: Any performance claims of the software been confirmed.
[x] Automated tests: Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
[x] Continuous Integration: Has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
[x] Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines.

For packages co-submitting to JOSS

[ ] The package has an obvious research application according to JOSS's definition in their submission requirements.

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

[ ] A short summary describing the high-level functionality of the software
[ ] Authors: A list of authors with their affiliations
[ ] A statement of need clearly stating problems the software is designed to solve and its target audience.
[ ] References: with DOIs for all those that have one (e.g. papers, datasets, software).

Final approval (post-review)

[ ] The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing: 1 hour

Review Comments

Hi Group 10 members,

Excellent work on the canadacovidmetrics package! Generally, the package is well produced and well documented, and it was easier to install the software for myself to use and explore. However, I just have some several specific remarks which you may want to consider for future improvements to this Python package:

Rather than including all the functions in a single python script, such that they all have to be accessed under the same namespace, it may be better to write a separate python script for each function. For example, such that it would be possible to import the functions like from canadacovidmetrics.get_vaccinations import get_vaccinations.
For the examples which utilised datetimes as arguments (e.g get_deaths), it may be more ideal to abstract the use of the functions from dt away from the user, and allow more simple arguments where they can simply specify a start/end date and then directly specify the number of days before or after than date as an integer.
The README appears to be missing a usage/example for the get_recoveries() function, which may be helpful to some users
I have tried running all the code in the Usage & Examples section, and the code related to plotting the get_vaccinations() data does not seem to work, and currently throws an error.
Seeing as the functions have quite a lot of overlapping use (ie, they are all for acquiring covid data concerning different metrics), it may be more user friendly to bundle them into a single function, with an additional argument to specify which metric (deaths, vaccinations, cases) to return, and instead add other functions for things such as plotting the output.

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

[x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

[x] A statement of need clearly stating problems the software is designed to solve and its target audience in README
[x] Installation instructions: for the development version of package and any non-standard dependencies in README
[ ] Vignette(s) demonstrating major functionality that runs successfully locally
[x] Function Documentation: for all user-facing functions
[ ] Examples for all user-facing functions
[x] Community guidelines including contribution guidelines in the README or CONTRIBUTING.
[x] Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a setup.py file or elsewhere.

Readme requirements The package meets the readme requirements below:

[x] Package has a README.md file in the root directory.

The README should include, from top to bottom:

[x] The package name
[x] Badges for continuous integration and test coverage, a repostatus.org badge, and any other badges. If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.)
[x] Short description of goals of package, with descriptive links to all vignettes (rendered, i.e. readable, cf the documentation website section) unless the package is small and there’s only one vignette repeating the README.
[x] Installation instructions
[ ] Any additional setup required (authentication tokens, etc)
[x] Brief demonstration usage
[x] Direction to more detailed documentation (e.g. your documentation files or website).
[x] If applicable, how the package compares to other similar packages and/or how it relates to other packages
[x] Citation information

Usability

[x] The documentation is easy to find and understand
[x] The need for the package is clear
[x] All functions have documentation and associated examples for use

Functionality

[x] Installation: Installation succeeds as documented.
[x] Functionality: Any functional claims of the software been confirmed.
[x] Performance: Any performance claims of the software been confirmed.
[x] Automated tests: Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
[x] Continuous Integration: Has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
[x] Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines.

Final approval (post-review)

[ ] The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing: 1hr

Review Comments

You did a good job with your package. While it is simple it makes the task of getting covid data easy and efficient. Below you will see some suggestions to improve your model in future releases!

When running the vaccination code in the readthedocs example, I got the error date_vaccine_complete was not found as a column name. I think this should call should be date not date_vaccine_complete.
When running the code that displays the daily case count by province in the readthedocs example, I get an error that says None of ['date_report'] are in the columns. As a user I am unsure what this error means and do not know how to easily fix it.
On the API reference page, it would be nice to have more complex examples of your functions. This would make it clearer to the user as to what functionalities they can manipulate.
It would be good to explore more edge cases in your functions with clear warnings for the user. For example, if I enter a date before the pandemic began or has not yet happened, I get the key error: Date. This error could be clearer for the user so they know what has gone wrong.
Your functions seem to perform similar tasks with different data values as the target. It would be nice if more functionality was added to the package to do initial data processing such as plotting or summary statistics. This would expand its applicability and create more interest in your package from a wider range of users.
In the README, it may be nice to move the python ecosystem section to the bottom of the document and keep all the information about your package together. Where the python ecosystem section is placed now, I am drawn to other covid-related packages before I have even finished learning about yours.
In your tests, there are many assert statements that don’t have a return message attached. For ease of testing, I would recommend adding in a short explanation for each assert statement to clarify what is going wrong in your code.

UBC-MDS / software-review-2022