Submission Group 24: numeric_edahelper(Python)

[hjw0703]

---

Submitting Author: Michelle Wang(@michelle-wms), Jiwei Hu (@hjw0703), Anupriya Srivastava (@Anupriya-Sri), Sam Quist(@squisty) Package Name: numeric_edahelper(Python) One-Line Description of Package: The numeric_edahelper package is a package that aims to streamline Exploratory Data Analysis, specifically for numeric data in datasets. Repository Link: https://github.com/UBC-MDS/numeric_edahelper Version submitted: v1.0.7 Editor: Michelle Wang(@michelle-wms), Jiwei Hu (@hjw0703), Anupriya Srivastava (@Anupriya-Sri), Sam Quist(@squisty) Reviewer 1: Ruben De la Garza Macias Reviewer 2: Jennifer Hoang Reviewer 3: Harry Chan Reviewer 4: Junrong Zhu

---

Description

• Data scientists often spend alot of time preprocessing data to extract useful parts for their analysis. The numeric_edahelper package is a package that aims to streamline Exploratory Data Analysis, specifically for numeric data in datasets. The goal is to simplify some common and repetitive tasks during EDA and data preprocessing for data analysts, as well as add value to their workflow by presenting some useful insights in a quick manner (just calling our functions), such as displaying highly-correlated variables and outliers.

Scope

• Please indicate which category or categories this package falls under:
  • [x] Data retrieval
  • [x] Data extraction
  • [x] Data munging
  • [ ] Data deposition
  • [ ] Reproducibility
  • [ ] Geospatial
  • [ ] Education
  • [ ] Data visualization*

* Please fill out a pre-submission inquiry before submitting a data visualization package. For more info, see notes on categories of our guidebook.

• Explain how the and why the package falls under these categories (briefly, 1-2 sentences):

The package includes functions which can complete the following tasks:

Display some useful statistics
Detect outliers
Deal with missing values
Check for correlation between features

• Who is the target audience and what are scientific applications of this package?
  Data scientists who often spend a lot of time preprocessing data to extract useful parts for their analysis.
• Are there other Python packages that accomplish the same thing? If so, how does yours differ? In the Python open-source ecosystem, there exists some useful packages that already tackle EDA and preprocessing, but our goal is to make it even more light-weighted, fast and specifically tailored to present numeric EDA insights. One popular and useful package that can generate EDA reports is pandas profiling.

  Technical checks

For details about the pyOpenSci packaging requirements, see our packaging guide. Confirm each of the following by checking the box. This package:

• [x] does not violate the Terms of Service of any service it interacts with.
• [x] has an OSI approved license.
• [x] contains a README with instructions for installing the development version.
• [x] includes documentation with examples for all functions.
• [x] contains a vignette with examples of its essential functions and uses.
• [x] has a test suite.
• [x] has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.

Publication options

• [ ] Do you wish to automatically submit to the Journal of Open Source Software? If so:

JOSS Checks

- [ ] The package has an **obvious research application** according to JOSS's definition in their [submission requirements][JossSubmissionRequirements]. Be aware that completing the pyOpenSci review process **does not** guarantee acceptance to JOSS. Be sure to read their submission requirements (linked above) if you are interested in submitting to JOSS. - [ ] The package is not a "minor utility" as defined by JOSS's [submission requirements][JossSubmissionRequirements]: "Minor ‘utility’ packages, including ‘thin’ API clients, are not acceptable." pyOpenSci welcomes these packages under "Data Retrieval", but JOSS has slightly different criteria. - [ ] The package contains a `paper.md` matching [JOSS's requirements][JossPaperRequirements] with a high-level description in the package root or in `inst/`. - [ ] The package is deposited in a long-term repository with the DOI: *Note: Do not submit your package separately to JOSS*

Are you OK with Reviewers Submitting Issues and/or pull requests to your Repo Directly?

This option will allow reviewers to open smaller issues that can then be linked to PR's rather than submitting a more dense text based review. It will also allow you to demonstrate addressing the issue via PR links.

• [x] Yes I am OK with reviewers submitting requested changes as issues to my repo. Reviewers will then link to the issues in their submitted review.

Code of conduct

• [x] I agree to abide by pyOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

P.S. *Have feedback/comments about our review process? Leave a comment here

Editor and Review Templates

Editor and review templates can be found here

[ruben1dlg]

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

• [x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

• [x] A statement of need clearly stating problems the software is designed to solve and its target audience in README
• [x] Installation instructions: for the development version of package and any non-standard dependencies in README
• [ ] Vignette(s) demonstrating major functionality that runs successfully locally
• [x] Function Documentation: for all user-facing functions
• [x] Examples for all user-facing functions
• [x] Community guidelines including contribution guidelines in the README or CONTRIBUTING.
• [x] Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a setup.py file or elsewhere.

Readme requirements The package meets the readme requirements below:

• [x] Package has a README.md file in the root directory.

The README should include, from top to bottom:

• [x] The package name
• [ ] Badges for continuous integration and test coverage, a repostatus.org badge, and any other badges. If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.)
• [x] Short description of goals of package, with descriptive links to all vignettes (rendered, i.e. readable, cf the documentation website section) unless the package is small and there’s only one vignette repeating the README.
• [x] Installation instructions
• [x] Any additional setup required (authentication tokens, etc)
• [x] Brief demonstration usage
• [x] Direction to more detailed documentation (e.g. your documentation files or website).
• [x] If applicable, how the package compares to other similar packages and/or how it relates to other packages
• [ ] Citation information

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole. Package structure should follow general community best-practices. In general please consider:

• [x] The documentation is easy to find and understand
• [x] The need for the package is clear
• [x] All functions have documentation and associated examples for use

Functionality

• [x] Installation: Installation succeeds as documented.
• [x] Functionality: Any functional claims of the software been confirmed.
• [x] Performance: Any performance claims of the software been confirmed.
• [x] Automated tests: Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
• [x] Continuous Integration: Has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
• [x] Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines.

For packages co-submitting to JOSS

• [ ] The package has an obvious research application according to JOSS's definition in their submission requirements.

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

• [ ] A short summary describing the high-level functionality of the software
• [ ] Authors: A list of authors with their affiliations
• [ ] A statement of need clearly stating problems the software is designed to solve and its target audience.
• [ ] References: with DOIs for all those that have one (e.g. papers, datasets, software).

Final approval (post-review)

• [ ] The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing:

1 hours

Review Comments

I like your package a lot guys! Here are some comments:

1. I would try to display the badges for both the docs and the codecov, just so it is easier for users to know the coverage and access the docs directly.
2. I see the example code for the usage in the readme file, but you only show the code. I think it would be a good idea to show how each of the functions modifies the data.
3. In your ReadTheDocs page, I see that you do not have an example usage, which kind of relates to the last point. It would be nice to see an example and how each one of the functions work.
4. I see that you have some open issues that are over a week old. I think it would be a good idea to close them of they are finished, just to have a better visibility of what is pending and who is assigned to it, as well as the deadline for each one.
5. I noticed that the name of some branches contain names of the collaborators. I would try to use more generic names like feat-something or fix-something, so that it is clearer for the rest of the collaborators.

[harryyikhchan]

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

• [x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

• [x] A statement of need clearly stating problems the software is designed to solve and its target audience in README
• [x] Installation instructions: for the development version of package and any non-standard dependencies in README
• [x] Vignette(s) demonstrating major functionality that runs successfully locally
• [x] Function Documentation: for all user-facing functions
• [ ] Examples for all user-facing functions
• [x] Community guidelines including contribution guidelines in the README or CONTRIBUTING.
• [x] Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a setup.py file or elsewhere.

Readme requirements The package meets the readme requirements below:

• [x] Package has a README.md file in the root directory.

The README should include, from top to bottom:

• [x] The package name
• [ ] Badges for continuous integration and test coverage, a repostatus.org badge, and any other badges. If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.)
• [x] Short description of goals of package, with descriptive links to all vignettes (rendered, i.e. readable, cf the documentation website section) unless the package is small and there’s only one vignette repeating the README.
• [x] Installation instructions
• [x] Any additional setup required (authentication tokens, etc)
• [x] Brief demonstration usage
• [x] Direction to more detailed documentation (e.g. your documentation files or website).
• [x] If applicable, how the package compares to other similar packages and/or how it relates to other packages
• [x] Citation information

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole. Package structure should follow general community best-practices. In general please consider:

• [x] The documentation is easy to find and understand
• [x] The need for the package is clear
• [ ] All functions have documentation and associated examples for use

Functionality

• [x] Installation: Installation succeeds as documented.
• [x] Functionality: Any functional claims of the software been confirmed.
• [x] Performance: Any performance claims of the software been confirmed.
• [x] Automated tests: Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
• [x] Continuous Integration: Has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
• [x] Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines.

For packages co-submitting to JOSS

• [ ] The package has an obvious research application according to JOSS's definition in their submission requirements.

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

• [ ] A short summary describing the high-level functionality of the software
• [ ] Authors: A list of authors with their affiliations
• [ ] A statement of need clearly stating problems the software is designed to solve and its target audience.
• [ ] References: with DOIs for all those that have one (e.g. papers, datasets, software).

Final approval (post-review)

• [ ] The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing: 1.5 hrs

---

Review Comments

Great jobs, everyone! Automating EDA is always a strong need for every data scientist. Below is my feedback for future improvement.

1. In your ReadtheDocs website, I think it would be more informative if you can include a good intro dataset such as penguins, iris datasets to demonstrate the functionality of all the package functions. So the users can relate to their own use cases before diving into any technical details.
2. I think it would be better to structure your test case in one particular datatype / error for each unit function, e.g. test_simple_df, test_multiple_df_raise_error. So the contributors can easily include more edge cases in the test suite.
3. It would be great if you can include more edge cases, such as different datatype, empty dataframe, a long list to see how the defensive programming mechanism reacts.
4. As a developer, I would also like to see the code coverage report. It's a good idea to include a codecov badge in README.md
5. I found that there is a missing example in the function overview(). It is suggested to include a simple example to guide the users.

[jennifer-hoang]

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

• [x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

• [x] A statement of need clearly stating problems the software is designed to solve and its target audience in README
• [x] Installation instructions: for the development version of package and any non-standard dependencies in README
• [ ] Vignette(s) demonstrating major functionality that runs successfully locally
• [x] Function Documentation: for all user-facing functions
• [x] Examples for all user-facing functions
• [x] Community guidelines including contribution guidelines in the README or CONTRIBUTING.
• [x] Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a setup.py file or elsewhere.

Readme requirements The package meets the readme requirements below:

• [x] Package has a [README.md](http://readme.md/) file in the root directory.

The README should include, from top to bottom:

• [x] The package name
• [ ] Badges for continuous integration and test coverage, a [repostatus.org](http://repostatus.org/) badge, and any other badges. If the README has many more badges, you might want to consider using a table for badges: [see this example](https://github.com/ropensci/drake). Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.)
• [x] Short description of goals of package, with descriptive links to all vignettes (rendered, i.e. readable, cf the documentation website section) unless the package is small and there’s only one vignette repeating the README.
• [x] Installation instructions
• [x] Any additional setup required (authentication tokens, etc)
• [x] Brief demonstration usage
• [x] Direction to more detailed documentation (e.g. your documentation files or website).
• [x] If applicable, how the package compares to other similar packages and/or how it relates to other packages
• [x] Citation information

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole. Package structure should follow general community best-practices. In general please consider:

• [x] The documentation is easy to find and understand
• [x] The need for the package is clear
• [ ] All functions have documentation and associated examples for use

Functionality

• [x] Installation: Installation succeeds as documented.
• [x] Functionality: Any functional claims of the software been confirmed.
• [x] Performance: Any performance claims of the software been confirmed.
• [x] Automated tests: Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
• [x] Continuous Integration: Has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
• [x] Packaging guidelines: The package conforms to the pyOpenSci [packaging guidelines](https://www.pyopensci.org/contributing-guide/authoring/index.html#packaging-guide).

For packages co-submitting to JOSS

• [ ] The package has an obvious research application according to JOSS's definition in their [submission requirements](http://joss.theoj.org/about#submission_requirements).

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching [JOSS's requirements](http://joss.theoj.org/about#paper_structure) with:

• [ ] A short summary describing the high-level functionality of the software
• [ ] Authors: A list of authors with their affiliations
• [ ] A statement of need clearly stating problems the software is designed to solve and its target audience.
• [ ] References: with DOIs for all those that have one (e.g. papers, datasets, software).

Final approval (post-review)

• [ ] The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing: 1 hr

---

Review Comments

Great work everyone! Please find a few suggestions for improvement below:

1. As mentioned by other reviewers, it would be useful for new users to have some examples in the Read the Docs files showcasing how to use the function and the expected outputs.
2. A code coverage badge or link to the test coverage report in the README would also be helpful.
3. There are a few very minor grammatical errors in the README that could be fixed to improve readability (”alot”, “light-weighted”).
4. In the documentation of the overview function, it would be beneficial to be more specific about what statistical values the function is able to calculate (mean, standard deviation, etc.), since a general user might not be able to immediately guess what values can be calculated.
5. The Read The Docs copyright information (in the conf.py file) only mentions one author, but it would be nice to include or mention all authors since you’ve all worked hard on this package together!

Best, Jennifer

[zjr-mds]

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

• [x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

• [x] A statement of need clearly stating problems the software is designed to solve and its target audience in README
• [x] Installation instructions: for the development version of package and any non-standard dependencies in README
• [ ] Vignette(s) demonstrating major functionality that runs successfully locally
• [x] Function Documentation: for all user-facing functions
• [x] Examples for all user-facing functions
• [x] Community guidelines including contribution guidelines in the README or CONTRIBUTING.
• [ ] Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a setup.py file or elsewhere.

Readme requirements The package meets the readme requirements below:

• [x] Package has a README.md file in the root directory.

The README should include, from top to bottom:

• [x] The package name
• [x] Badges for continuous integration and test coverage, a repostatus.org badge, and any other badges. If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.)
• [x] Short description of goals of package, with descriptive links to all vignettes (rendered, i.e. readable, cf the documentation website section) unless the package is small and there’s only one vignette repeating the README.
• [x] Installation instructions
• [x] Any additional setup required (authentication tokens, etc)
• [x] Brief demonstration usage
• [ ] Direction to more detailed documentation (e.g. your documentation files or website).
• [x] If applicable, how the package compares to other similar packages and/or how it relates to other packages
• [ ] Citation information

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole. Package structure should follow general community best-practices. In general please consider:

• [x] The documentation is easy to find and understand
• [x] The need for the package is clear
• [x] All functions have documentation and associated examples for use

Functionality

• [x] Installation: Installation succeeds as documented.
• [x] Functionality: Any functional claims of the software been confirmed.
• [x] Performance: Any performance claims of the software been confirmed.
• [x] Automated tests: Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
• [x] Continuous Integration: Has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
• [x] Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines.

For packages co-submitting to JOSS

• [ ] The package has an obvious research application according to JOSS's definition in their submission requirements.

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

• [ ] A short summary describing the high-level functionality of the software
• [ ] Authors: A list of authors with their affiliations
• [ ] A statement of need clearly stating problems the software is designed to solve and its target audience.
• [ ] References: with DOIs for all those that have one (e.g. papers, datasets, software).

Final approval (post-review)

• [ ] The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing: 1 hour

Congratulations on creating this amazing package! Great work! All functions worked properly as expected from the repository description and I only had a few minor suggestions as below! Cheers!

1. Edit the ‘About' description to make the repository look nicer. Maybe you can also add the link to read_the_doc under the description in ‘About’.
2. Minor typo on readme: missing space in “alot ”
3. Removing some old branches to avoid possible confusion; I noticed that you have over 10 branches but some of them are more than 40+ commits behind the main branch. In this case, it’d be okay to remove the ones that you don't use anymore.
4. More tests can be added for overview function to check the sum of other columns from the output dataframe.
5. Also, the example.ipynb is not updated to the most recent version like what’s shown from readthedoc, maybe you could take a look into that.

Review Comments