search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
719 stars 38 forks source link

[REVIEW]: FuzzyClass: An R package for Fuzzy and Non-Fuzzy probabilistic-based classifiers #5613

Closed editorialbot closed 1 year ago

editorialbot commented 1 year ago

Submitting author: !--author-handle-->@Jodavid<!--end-author-handle-- (Jodavid Ferreira) Repository: https://github.com/leapigufpb/FuzzyClass/ Branch with paper.md (empty if default branch): paper Version: v0.1.5 Editor: !--editor-->@ajstewartlang<!--end-editor-- Reviewers: @tianyi93, @MikeLydeamore Archive: 10.5281/zenodo.8280775

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/5352f4779dbc130496348642554833b0"><img src="https://joss.theoj.org/papers/5352f4779dbc130496348642554833b0/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/5352f4779dbc130496348642554833b0/status.svg)](https://joss.theoj.org/papers/5352f4779dbc130496348642554833b0)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@tianyi93 & @Athene-ai, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @ajstewartlang know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @tianyi93

📝 Checklist for @MikeLydeamore

editorialbot commented 1 year ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 1 year ago 
Software report:

github.com/AlDanial/cloc v 1.88  T=0.08 s (552.4 files/s, 207384.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JavaScript                       3           2075           1917           6898
R                               19            268           1409           1882
HTML                            10            275             33            942
TeX                              2             75              5            490
Markdown                         5             84              0            331
XML                              1              0              0             84
Rmd                              1             33             40             24
YAML                             2              2              4             21
CSS                              1              0              0              1
JSON                             1              0              0              1
-------------------------------------------------------------------------------
SUM:                            45           2812           3408          10674
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository
editorialbot commented 1 year ago

Wordcount for paper.md is 1218

editorialbot commented 1 year ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/0304-4076(86)90002-3 is OK
- 10.1007/978-1-4899-3242-6 is OK
- 10.18637/jss.v023.i07 is OK
- 10.1007/978-0-387-21706-2 is OK
- 10.18637/jss.v032.i10 is OK
- 10.18637/jss.v027.i08 is OK

MISSING DOIs

- 10.1093/comjnl/bxm073 may be a valid DOI for title: Computational intelligence: principles, techniques and applications
- 10.1016/s0020-0255(02)00154-8 may be a valid DOI for title: Fuzzy expert systems architecture for image classification using mathematical morphology operators
- 10.1007/978-3-540-87481-2_13 may be a valid DOI for title: A genetic algorithm for text classification rule induction
- 10.1016/j.knosys.2014.05.006 may be a valid DOI for title: Psychomotor skills assessment in medical training based on virtual reality using a weighted possibilistic approach
- 10.1142/9789814619998_0029 may be a valid DOI for title: Accuracy evaluation of evolving fuzzy neural networks in pattern recognition tasks using data from four different statistical distributions
- 10.3233/jifs-179431 may be a valid DOI for title: A double weighted fuzzy gamma naive bayes classifier
- 10.1109/fuzz-ieee.2016.7737762 may be a valid DOI for title: A Fuzzy Binomial Naive Bayes classifier for epidemiological data
- 10.1088/0026-1394/44/2/003 may be a valid DOI for title: Trapezoidal and triangular distributions for Type B evaluation of standard uncertainty
- 10.1111/j.1442-9993.2010.02180.x may be a valid DOI for title: A Practical Guide to Ecological Modelling. Using R as a Simulation Platform 
- 10.1142/9789811223334_0054 may be a valid DOI for title: Online skills assessment in training based on virtual reality using a novel fuzzy triangular naive Bayes network
- 10.1007/s40815-020-00936-4 may be a valid DOI for title: A New Bayesian Network Based on Gaussian Naive Bayes with Fuzzy Parameters for Training Assessment in Virtual Simulators
- 10.24042/djm.v4i1.7737 may be a valid DOI for title: Confidence interval estimation of gamma distribution lifetime data using score and bootstrap methods
- 10.1142/9789813146976_0035 may be a valid DOI for title: A fuzzy exponential naive bayes classifier
- 10.5220/0005642801930198 may be a valid DOI for title: A fuzzy poisson naive bayes classifier for epidemiological purposes
- 10.1109/fuzz-ieee.2016.7737762 may be a valid DOI for title: A Fuzzy Binomial Naive Bayes classifier for epidemiological data
- 10.1142/9789811269264_0071 may be a valid DOI for title: A new Fuzzy Trapezoidal Naive Bayes Network as basis for assessment in training based on virtual reality

INVALID DOIs

- None
editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

ajstewartlang commented 1 year ago

@editorialbot remove @Athene-ai from reviewers

Sorry @Athene-ai but given some information I've received I'll need to remove you from this submission.

editorialbot commented 1 year ago

@Athene-ai removed from the reviewers list!

Athene-ai commented 1 year ago

As you wish @ajstewartlang

ajstewartlang commented 1 year ago

:wave: @tbsexton might you be able to review this submission please?

ajstewartlang commented 1 year ago

:wave: @MikeLydeamore might you have the bandwidth to review this submission please?

tianyizhangcs commented 1 year ago

Thanks for the invitation @ajstewartlang, will review it in next couple days.

tianyizhangcs commented 1 year ago

Review checklist for @tianyi93

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

MikeLydeamore commented 1 year ago

Hi @ajstewartlang I can take this on.

editorialbot commented 1 year ago

@MikeLydeamore I can't do that because you are not a reviewer

ajstewartlang commented 1 year ago

@editorialbot add @MikeLydeamore as reviewer

editorialbot commented 1 year ago

@MikeLydeamore added to the reviewers list!

MikeLydeamore commented 1 year ago

Review checklist for @MikeLydeamore

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

tianyizhangcs commented 1 year ago

@editorialbot commands

editorialbot commented 1 year ago

Hello @tianyi93, here are the things you can ask me to do:


# List all available commands
@editorialbot commands

# Get a list of all editors's GitHub handles
@editorialbot list editors

# Check the references of the paper for missing DOIs
@editorialbot check references

# Perform checks on the repository
@editorialbot check repository

# Adds a checklist for the reviewer using this command
@editorialbot generate my checklist

# Set a value for branch
@editorialbot set joss-paper as branch

# Generates the pdf paper
@editorialbot generate pdf

# Generates a LaTeX preprint file
@editorialbot generate preprint

# Get a link to the complete list of reviewers
@editorialbot list reviewers
tianyizhangcs commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

MikeLydeamore commented 1 year ago

Hi @ajstewartlang @tianyi93. I have completed my review of this work. The paper itself is well written, and bar one typo (exponencial distribution), the only thing missing is a statement of other software solutions that exist for this problem.

I believe there are some issues with the codebase and repository which should be amended before publication:

Finally, the example uses a function GauNBFuzzyParam. The documentation for this has a few issues:

tianyizhangcs commented 1 year ago

I have completed my review of this work. Few issues would suggest to amend before publication:

Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.

Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?

Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?`

State of the field: Do the authors describe how this software compares to other commonly-used packages?

  • [ ] there is no explicit description of how the "FuzzyClass" software compares to other commonly-used packages for classification tasks. The text primarily focuses on introducing the concept of Naive Bayes and Fuzzy Naive Bayes classifiers, their probability calculations, and the motivating example using the "FuzzyClass" package with real data.
ajstewartlang commented 1 year ago

Thanks for your super helpful comments @MikeLydeamore and @tianyi93 - they're very much appreciated.

:wave: @Jodavid Could you address the above suggestions please?

Jodavid commented 1 year ago

Yes, @ajstewartlang, of course.

Thank you, @MikeLydeamore and @tianyi93, for the suggestions.

I will incorporate them into the paper.

Jodavid commented 1 year ago

Hello @ajstewartlang, @MikeLydeamore, and @tianyi93, returning with responses to suggestions

Below are each of them with responses regarding what has been accomplished. I have added a checkmark and a response to each suggestion.

@MikeLydeamore:

Hi @ajstewartlang @tianyi93. I have completed my review of this work. [x] The paper itself is well written, and bar one typo (exponencial distribution), Response: It has been rectified.

[X] the only thing missing is a statement of other software solutions that exist for this problem. Response: In the final paragraph of the introduction, information has been included regarding other libraries and software that encompass certain methods found within FuzzyClass. However, the majority of solutions, particularly those involving fuzzy probability, are exclusively available in FuzzyClass.

I believe there are some issues with the codebase and repository which should be amended before publication:

[X] Please fix the pkgdown site: https://leapigufpb.github.io/FuzzyClass/reference/index.html gives me a 404 error Response: It has been rectified.

[X] Some automated testing of the software (beyond the current spellcheck) would be a great addition to robustness. Response: The test_type.R file has been included as a classifiers test.

[X] Please add at least one vignette. This is a requirement for CRAN these days. Response: A file containing an example of data analysis and instructions on how to utilize the classifier has been generated.

[X] Please add contribution guidelines to the README Response: This section has been added to the README (https://github.com/leapigufpb/FuzzyClass/).

[X] Please add issue reporting guidelines to the README Response: This section has been added to the README (https://github.com/leapigufpb/FuzzyClass/)

[X] The example in the README could do with much stronger documentation/commenting. It runs successfully but I don't know what it's supposed to do, or what the outputs mean. Response: Steps regarding the code, problem statement, and usage example have been added.

[X] Dependencies: The caret library is used in the example, but this isn't included in the package install. It's a massive library, so I understand not wanting to include it as a dependency, but either an alternative confusionMatrix function should be sourced, or it needs to go into DEPENDS. Response: Indeed, the caret package was employed for generating the confusion matrix. I also comprehend the rationale behind constructing an alternative confusion matrix. However, given the robustness of the confusionMatrix function in caret package, the authors opted to omit the caret-related portion from the example, instead opting for a simpler approach. Those desiring more resilient outcomes can avail themselves of the confusionMatrix function within caret.

Finally, the example uses a function GauNBFuzzyParam. The documentation for this has a few issues:

[X] Parameter w documentations talks about metd == 4, but the documentation about metd only says this can range between 1 and 3 Response: Given that this solution is still under development, the maintainers have chosen to exclude the metd=4 from the code.

[X] Argument cores isn't used in this function (passed through elsewhere), and this should be stated as I was looking for the parallel pool to spawn in this function, not elsewhere. Response: The 'cores' argument is employed within the prediction function, owing to the computational costliness of these methods during prediction, rather than parameter estimation. A comment has been appended to the line concerning 'cores' within the manual of all functions, elucidating this detail.

@tianyi93:

I have completed my review of this work. Few issues would suggest to amend before publication:

Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.

[X] Readme: please include running the command from R console Response: The list of dependencies has been appended to the README; however, it is automatically installed during the package installation in R. The commands are present in .Rmd files, which are executed in R before generating the README. Moreover, an image of the R console displaying the installation and package loading commands has been incorporated.

Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

[X] No clear guidelines for third parties wishing to contribute to the software or seek support Response: This section has been added to the README (https://github.com/leapigufpb/FuzzyClass/).

Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?

[X] Did not see much core functionality of the software documented Response: The program's documentation/manual is available at the following link: https://cran.r-project.org/web/packages/FuzzyClass/FuzzyClass.pdf. Additionally, examples have been included in the README, and a Vignette has been created to provide usage examples.

Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?`

[X] Please add the above Response: The test_type.R file has been included as a classifiers test.

State of the field: Do the authors describe how this software compares to other commonly-used packages?

[X] there is no explicit description of how the "FuzzyClass" software compares to other commonly-used packages for classification tasks. The text primarily focuses on introducing the concept of Naive Bayes and Fuzzy Naive Bayes classifiers, their probability calculations, and the motivating example using the "FuzzyClass" package with real data. Response: In the final paragraph of the introduction, information has been included regarding other libraries and software that encompass certain methods found within FuzzyClass. However, the majority of solutions, particularly those involving fuzzy probability, are exclusively available in FuzzyClass.

ajstewartlang commented 1 year ago

Thanks for addressing these, @Jodavid.

:wave: @MikeLydeamore and @tianyi93 could you take a look please and let me know whether these additions/edits/clarifications address your issues?

tianyizhangcs commented 1 year ago

LGTM, no other major comments. Would be good if can make The program's documentation/manual easier for users to find. https://cran.r-project.org/web/packages/FuzzyClass/FuzzyClass.pdf.

ajstewartlang commented 1 year ago

LGTM, no other major comments. Would be good if can make The program's documentation/manual easier for users to find. https://cran.r-project.org/web/packages/FuzzyClass/FuzzyClass.pdf.

@Jodavid might you be able to address this please?

Jodavid commented 1 year ago

Hello @ajstewartlang, of course,

The link to the manual was already present in the paper in the last paragraph before the 'Acknowledgements'.

It has also been added to the readme at https://github.com/leapigufpb/FuzzyClass/ before the 'Dependencies' section with the following text:

"A package manual that showcases the existing classifiers and demonstrates how to use it can be found at the following link: https://cran.r-project.org/web/packages/FuzzyClass/FuzzyClass.pdf"

Thanks for suggestion @tianyi93

ajstewartlang commented 1 year ago

Many thanks for adding that @Jodavid

:wave: @MikeLydeamore are you happy with the additions/clarifications etc?

MikeLydeamore commented 1 year ago

I am happy. Thanks for the quick turnaround.

ajstewartlang commented 1 year ago

@editorialbot check references

ajstewartlang commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1016/0304-4076(86)90002-3 is OK
- 10.1007/978-1-4899-3242-6 is OK
- 10.18637/jss.v023.i07 is OK
- 10.1007/978-0-387-21706-2 is OK
- 10.18637/jss.v032.i10 is OK
- 10.18637/jss.v027.i08 is OK

MISSING DOIs

- 10.1093/comjnl/bxm073 may be a valid DOI for title: Computational intelligence: principles, techniques and applications
- 10.1016/s0020-0255(02)00154-8 may be a valid DOI for title: Fuzzy expert systems architecture for image classification using mathematical morphology operators
- 10.1007/978-3-540-87481-2_13 may be a valid DOI for title: A genetic algorithm for text classification rule induction
- 10.1016/j.knosys.2014.05.006 may be a valid DOI for title: Psychomotor skills assessment in medical training based on virtual reality using a weighted possibilistic approach
- 10.1142/9789814619998_0029 may be a valid DOI for title: Accuracy evaluation of evolving fuzzy neural networks in pattern recognition tasks using data from four different statistical distributions
- 10.3233/jifs-179431 may be a valid DOI for title: A double weighted fuzzy gamma naive bayes classifier
- 10.1109/fuzz-ieee.2016.7737762 may be a valid DOI for title: A Fuzzy Binomial Naive Bayes classifier for epidemiological data
- 10.1088/0026-1394/44/2/003 may be a valid DOI for title: Trapezoidal and triangular distributions for Type B evaluation of standard uncertainty
- 10.1111/j.1442-9993.2010.02180.x may be a valid DOI for title: A Practical Guide to Ecological Modelling. Using R as a Simulation Platform 
- 10.1142/9789811223334_0054 may be a valid DOI for title: Online skills assessment in training based on virtual reality using a novel fuzzy triangular naive Bayes network
- 10.1007/s40815-020-00936-4 may be a valid DOI for title: A New Bayesian Network Based on Gaussian Naive Bayes with Fuzzy Parameters for Training Assessment in Virtual Simulators
- 10.24042/djm.v4i1.7737 may be a valid DOI for title: Confidence interval estimation of gamma distribution lifetime data using score and bootstrap methods
- 10.1142/9789813146976_0035 may be a valid DOI for title: A fuzzy exponential naive bayes classifier
- 10.5220/0005642801930198 may be a valid DOI for title: A fuzzy poisson naive bayes classifier for epidemiological purposes
- 10.1109/fuzz-ieee.2016.7737762 may be a valid DOI for title: A Fuzzy Binomial Naive Bayes classifier for epidemiological data
- 10.1142/9789811269264_0071 may be a valid DOI for title: A new Fuzzy Trapezoidal Naive Bayes Network as basis for assessment in training based on virtual reality

INVALID DOIs

- None
editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

ajstewartlang commented 1 year ago

@Jodavid could you add the appropriate DOIs to the outputs in the reference list please? The suggested ones are above, but it would be worth checking that each is correct.

Jodavid commented 1 year ago

Hello @ajstewartlang,

I've checked all the DOIs, added the missing ones, and confirmed and corrected the existing ones. The files are already updated in the repository.

ajstewartlang commented 1 year ago

@editorialbot check references

editorialbot commented 1 year ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1093/comjnl/bxm073 is OK
- 10.1016/s0020-0255(02)00154-8 is OK
- 10.1002/9780470872352.fmatter is OK
- 10.1007/978-3-540-87481-2_13 is OK
- 10.1016/j.knosys.2014.05.006 is OK
- 10.1142/9789814619998_0029 is OK
- 10.3233/jifs-179431 is OK
- 10.1109/fuzz-ieee.2016.7737762 is OK
- 10.1088/0026-1394/44/2/003 is OK
- 10.1016/s0075-9511(09)00026-7 is OK
- 10.1142/9789811223334_0054 is OK
- 10.1007/s40815-020-00936-4 is OK
- 10.24042/djm.v4i1.7737 is OK
- 10.1142/9789813146976_0035 is OK
- 10.5220/0005642801930198 is OK
- 10.1109/fuzz-ieee.2016.7737762 is OK
- 10.1016/0304-4076(86)90002-3 is OK
- 10.1201/9780203753736 is OK
- 10.18637/jss.v023.i07 is OK
- 10.1007/978-0-387-21706-2 is OK
- 10.18637/jss.v032.i10 is OK
- 10.18637/jss.v027.i08 is OK
- 10.1142/9789811269264_0071 is OK

MISSING DOIs

- None

INVALID DOIs

- None
ajstewartlang commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

ajstewartlang commented 1 year ago

@editorialbot set paper as branch

ajstewartlang commented 1 year ago

@editorialbot set paper as branch

editorialbot commented 1 year ago

Done! branch is now paper

ajstewartlang commented 1 year ago

@editorialbot check references

editorialbot commented 1 year ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1093/comjnl/bxm073 is OK
- 10.1016/s0020-0255(02)00154-8 is OK
- 10.1002/9780470872352.fmatter is OK
- 10.1007/978-3-540-87481-2_13 is OK
- 10.1016/j.knosys.2014.05.006 is OK
- 10.1142/9789814619998_0029 is OK
- 10.3233/jifs-179431 is OK
- 10.1109/fuzz-ieee.2016.7737762 is OK
- 10.1088/0026-1394/44/2/003 is OK
- 10.1016/s0075-9511(09)00026-7 is OK
- 10.1142/9789811223334_0054 is OK
- 10.1007/s40815-020-00936-4 is OK
- 10.24042/djm.v4i1.7737 is OK
- 10.1142/9789813146976_0035 is OK
- 10.5220/0005642801930198 is OK
- 10.1109/fuzz-ieee.2016.7737762 is OK
- 10.1016/0304-4076(86)90002-3 is OK
- 10.1201/9780203753736 is OK
- 10.18637/jss.v023.i07 is OK
- 10.1007/978-0-387-21706-2 is OK
- 10.18637/jss.v032.i10 is OK
- 10.18637/jss.v027.i08 is OK
- 10.1142/9789811269264_0071 is OK

MISSING DOIs

- None

INVALID DOIs

- None
ajstewartlang commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

ajstewartlang commented 1 year ago

@Jodavid there still seem to be some DOIs missing (e.g., for this one: https://www.worldscientific.com/doi/abs/10.1142/9789812774118_0111) - they're not being picked up automatically - could you add those missing ones manually please?