search

UBC-MDS / software-review-2022

0 stars 0 forks source link

Submission group 12: simplerfit(R) #49

Open zzhzoe opened 2 years ago

zzhzoe commented 2 years ago

Submitting Author Name:Zihan Zhou
Submitting Author Github Handle: @zzhzoe Other Package Authors Github handles: Mohammadreza Mirzazadeh @rezam747 Navya Dahiya @nd265 Sanchit Singh @Sanchit120496 Repository: https://github.com/UBC-MDS/simplerfit Version submitted: Standard Submission type: Standard Reviewers: Abhiket Gaurav @Abhiket, Sufang Tan @Kendy-Tan, Lakshmi Santosha Valli Akella @valli180, Pavel Levchenko @plevchen Archive: TBD Version accepted: TBD Language: en

Package: simplerfit
Title: Clean data, perform EDA, fit classifier or regressor models and return model performance scores
Version: 0.0.0.9000
Authors@R: 
    c(person(given = "Navya ",
           family = "Dahiya",
           role = c("aut", "cre"),
           email = "navyad265@gmail.com",
           ),
    person(given = "Reza",
           family = "Mirzazadeh",
           role = "ctb",
           ),
    person(given = "Sanchit",
           family = "Singh",
           role = "ctb"),
           person(given = "Zhian",
           family = "Zoe",
           role = "ctb"))
Description: This package helps data scientists to clean the data, perform basic EDA, visualize graphical interpretations and analyse performance of the baseline model and basic Classification or Regression models, namely Logistic Regression, Ridge on their data.
License: MIT + file LICENSE
Encoding: UTF-8
LazyData: true
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.1.2
Suggests: 
    covr,
    testthat (>= 3.0.0)
Config/testthat/edition: 3
Imports: 
    readr,
    caret,
    mltools,
    GGally,
    ggplot2,
    data.table,
    devtools,
    tidyr,
    gapminder,
    dplyr,
    stringr,
    rlang,
    stats,
    monomvn

Scope

A R package that cleans the data, does basic EDA and returns scores for basic classification and regression models. This package helps data scientists to clean the data, perform basic EDA, visualize graphical interpretations and analyse performance of the baseline model and basic Classification or Regression models, namely Logistic Regression, Ridge on their data.

Any data professionals at the entry-level who would like to conduct a quick exploratory data analysis. A data scientist spends a lot of time writing same syntactical code for carrying out data processing, transformations, fitting models and comparing their performances.

Technical checks

Confirm each of the following by checking the box.

This package:

Publication options

MEE Options - [ ] The package is novel and will be of interest to the broad readership of the journal. - [ ] The manuscript describing the package is no longer than 3000 words. - [ ] You intend to archive the code for the package in a long-term repository which meets the requirements of the journal (see [MEE's Policy on Publishing Code](http://besjournals.onlinelibrary.wiley.com/hub/journal/10.1111/(ISSN)2041-210X/journal-resources/policy-on-publishing-code.html)) - (*Scope: Do consider MEE's [Aims and Scope](http://besjournals.onlinelibrary.wiley.com/hub/journal/10.1111/(ISSN)2041-210X/aims-and-scope/read-full-aims-and-scope.html) for your manuscript. We make no guarantee that your manuscript will be within MEE scope.*) - (*Although not required, we strongly recommend having a full manuscript prepared when you submit here.*) - (*Please do not submit your package separately to Methods in Ecology and Evolution*)

Code of conduct

abhiket commented 2 years ago

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

Documentation

The package includes all the following forms of documentation:

Readme requirements The package meets the readme requirements below:

The README should include, from top to bottom:

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole. Package structure should follow general community best practices. In general please consider:

Functionality

For packages co-submitting to JOSS

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

Final approval (post-review)

Estimated hours spent reviewing: 1.5hrs

Review Comments

Going through the package I am amazed by the ideation and conceptualization of this project. I am sure this package has lots of practical applications. However, I think there are still a lot of unanswered areas and the user as of now has to use this package with a pinch of salt.

Similar to my review of the Python version, I feel that the followings are the areas where we need to pay attention:

  1. The scope and usability of this package could be more explicit. Also a few sentences on dummy classifier or dummy regressor which we are using as a baseline should be explained.
  2. Removing all the rows that contain missing values is not an ideal way to deal with missing values. As we are now aware that there are various ways of doing missing value imputations, some of them can be implemented here.
  3. The vignettes need to be more elaborate and self-explanatory. Right now it does not seem to explain in detail. The function of cleaning the dataset is not very clear. Can it be used to clean the data directly taken from a URL? Or the user has to remove the header and footer and then call the function to clean the dataset.
  4. Choice of choosing the plot type (histogram, scatter, box, etc) can be given as an input.
  5. Since we are doing EDA, we need to slice and dice the data, the option of faceting can also be added.
  6. The correlation plot does not seem to be appearing properly.
  7. The function here is using logistic regression for binary classification. Request to kindly mention it in the scope of the project in the README section.
  8. For the case in point, accuracy seems to be a good metric, as a user, one may not want to use accuracy as a metric. The choice of metric can also be an input to the function.

Overall, I was able to install the package and use it on a few toy datasets. All the functions within the package are working well and as per the expectation. However, I feel the documentation needs to be more elaborate, and self-explanatory even to a naive user. Given the time constraint, I have huge respect for the team. I am sure if developed to the fullest, this package can rock the data-science world!!

Kendy-Tan commented 2 years ago

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

Documentation

The package includes all the following forms of documentation:

Readme requirements The package meets the readme requirements below:

The README should include, from top to bottom:

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole. Package structure should follow general community best-practices. In general please consider:

Functionality

For packages co-submitting to JOSS

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

Final approval (post-review)

Estimated hours spent reviewing: 1.5 hrs

Review Comments

Good job team! Thank you for coming up with such a useful package, and I can't wait to use it in my real later project. however, I still have a few suggestions to make this function more versatile:

  1. The cleaner function can include more functions like data imputation for numerical columns by replacing the missing value with mean/median/ most frequent value.
  2. Both the classification and regression function could add more model options like decision trees and other non-linear terms been to add in regression.
  3. In the EAD plotting, you can faceting to see the correlations among different categorical groups. Although there is the docs badge, it would be great if the link is given in the "About" section.
  4. You can add more options in the plot type like histogram, scatter, the box that allows users to choose the most appropriate one for their investigation problem.
  5. If the package is limited to a typical structure of the data set, you should remind the user in the README.md.
plevchen commented 2 years ago

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

Documentation

The package includes all the following forms of documentation:

Functionality

Estimated hours spent reviewing: 1 hour

Review Comments

Nice work! Some of those could be useful for actual EDA work in the future. Some thoughts on potential improvements:

  1. Maybe the structure of the package should be more focused on either visualization or metrics. Combining both in the same package could be confusing to the end user
  2. In fit_regressor function users do not have options to select type of regression. Better approach could be assign some type as default in function definition, but provide a user an opportunity to select from multiple options
  3. Similar comment for fit_classifier function. A lot of calculations are hardcoded, not being flexible for user needs
  4. Cleaning function is too restrictive. This could be significantly modified with calculating number of missing observations, with making assumption whether that missingness can be treated as MAR, MNAR or MCAR with using different imputation methods.
  5. For tests, I think better option is to use small tibbles instead of full csv. It is much easier to spot what is the issue in such case, rather than working with a complex dataset

Great work with defensive programming. All functions were very well thought in terms of edge cases and what warnings/errors should be raised