openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
714 stars 38 forks source link

[REVIEW]: genderstat: An R Package for Comprehensive Gender Inequality Analysis #5932

Closed editorialbot closed 2 months ago

editorialbot commented 12 months ago

Submitting author: !--author-handle-->@mashrur-ayon<!--end-author-handle-- (S M Mashrur Arafin Ayon) Repository: https://github.com/mashrur-ayon/genderstat Branch with paper.md (empty if default branch): Version: 0.1.3 Editor: !--editor-->@spholmes<!--end-editor-- Reviewers: @thegargiulian, @joaojcorreia Archive: Pending

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/cade05e5f52b64e0e866ce27921bd041"><img src="https://joss.theoj.org/papers/cade05e5f52b64e0e866ce27921bd041/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/cade05e5f52b64e0e866ce27921bd041/status.svg)](https://joss.theoj.org/papers/cade05e5f52b64e0e866ce27921bd041)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@thegargiulian & @joaojcorreia, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @spholmes know.

Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest

Checklists

📝 Checklist for @thegargiulian

📝 Checklist for @joaojcorreia

editorialbot commented 12 months ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 12 months ago
Software report:

github.com/AlDanial/cloc v 1.88  T=0.05 s (512.2 files/s, 29993.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
R                                9            144            381            311
Markdown                         4            134              0            268
XML                             10              0              0            144
TeX                              1              5              0             82
YAML                             1              2              0             18
Rmd                              1             23             33             16
Python                           1              5              1             14
-------------------------------------------------------------------------------
SUM:                            27            313            415            853
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository
editorialbot commented 12 months ago

Wordcount for paper.md is 1710

editorialbot commented 12 months ago
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1080/1464988032000125773 is OK
- 10.7910/DVN/GZLYLI is OK

MISSING DOIs

- None

INVALID DOIs

- None
editorialbot commented 12 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

thegargiulian commented 12 months ago

Review checklist for @thegargiulian

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

joaojcorreia commented 12 months ago

Review checklist for @joaojcorreia

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

joaojcorreia commented 11 months ago

The paper should have a clearer summary, focusing on the library's purpose and its original data sources. The statement of need needs to better explain the package's benefits and be more transparent about data sources. The article lacks context about the field and how GenderStat fits in. The writing style should be simpler, and function descriptions clearer. The documentation should provide more detail on example usage, parameter explanations, and function functionality. It should also clarify data origin and include a link to CRAN documentation.

Bellow you will find my detailed review notes.


Notes on the article:

Summary:

Statement of need:

Assessment of the state of the field:

Quality of the writing:

References:

General comments:

Notes on the documentation:

General comments:

Statement of need:

Installation instructions:

Example usage:

Functionality documentation:

Automates tests:

Community guidelines:

mashrur-ayon commented 11 months ago

Dear Joäo Granja-Correia, @joaojcorreia

Firstly, I would like to express my sincere gratitude for your thorough and insightful review. One of the aspects I truly appreciate about the Journal of Open Source Software is the constructive feedback provided by its reviewers, and your comments exemplify this.

I acknowledge all the points you've raised and am committed to making the necessary revisions to address them. I'll also wait for feedback from the second reviewer before finalizing my edits, to ensure a comprehensive revision. Please bear with me as I take the time to ensure that each point is attended to adequately. If I find myself in need of further clarification or suggestions, I won't hesitate to reach out.

Once again, thank you for your guidance and support.

Warm regards, Mashrur.

thegargiulian commented 11 months ago

Hi all, just a note to say that I've got this review on my list for this upcoming week. More soon.

thegargiulian commented 10 months ago

Hi @mashrur-ayon, I hope you're well!

I've opened a couple of issues related to specific parts of the package. I agree largely with @joaojcorreia's comments, in particular, the paper and the package would benefit from reframing to highlight the project's main contribution—the data itself.

Notes about the paper:

Notes about the package:

arfon commented 8 months ago

:wave: @spholmes – how are things going here? This review seems to have stalled since ~November.

spholmes commented 8 months ago

Agreed @arfon , I was waiting for the author to step in and make the changes requested by the reviewers, I'll ping them now to see where they stand on either making the changes or withdrawing the paper.

spholmes commented 8 months ago

Hi @mashrur-ayon, I hope you're well!

I've opened a couple of issues related to specific parts of the package. I agree largely with @joaojcorreia's comments, in particular, the paper and the package would benefit from reframing to highlight the project's main contribution—the data itself.

Notes about the paper:

  • The paper and documentation should focus more on the provenance of the data: where did the data come from?, how and why was it created?, how was it processed by you for the purposes of this package? I would also recommend more carefully motivating why we need this package because that is not entirely clear from the statement of need in its current form.
  • The paper needs a more thorough survey of the field. While genderstat might be the first R package dedicated to making metrics related to the study of gender (in)equality, the paper would benefit from the discussion other attempts to elucidate these issues, even if they haven't resulted in the publication of an open source R package.
  • It would be helpful to provide more information about the core theoretical concepts you mention, especially how they have been applied or used, especially in social science research in the past. I'd also recommend a short discussion of why you chose these particular metrics as other metrics of gender equality do exist.
  • Generally, the language used in the paper is often overly complex and should be simplified. This should improve overall readability and reduce the word count.

Notes about the package:

  • I would consider restructuring or eliminating the graphing functions from the package. In their current form, the graphing functions do not provide users any flexibility to customize the graphics in the way that most suits their needs (e.g., plot_gdi() always plots the bottom 15 countries according to GDI). Instead of offering users these pre-built graphing functions, I would instead suggest providing some simple worked examples of real world social science research questions that users would be able to examine graphically using the data provided in the genderstat package. These types of examples would also improve the overall quality of the documentation.
  • Some of the datasets/metrics have missing values in them. I'd recommend commenting on this in the data or function documentation.
  • The data does not provide any time information, e.g., what year are the observations from? Without this information there is little context for the data. Additionally ,it appears that data is only available for a single year. I think the scholarly contribution of this package would be augmented by including a longer time series of information from the different sources you draw from. This would also provide crucial functionality for users who may be interested in how these metrics change over time.

Hi @mashrur-ayon , I wondered whether you could respond to the revisions requested by the reviewers? If you are not able to respond, how would like to proceed, do you want to withdraw at this time? Thanks, Susan

mashrur-ayon commented 8 months ago

Hello @spholmes ,

I hope this message finds you well. Please accept my apologies for the delayed response. My day-to-day activities have been quite demanding, and in addition, I have recently started a new position as a researcher at a university. This has impacted my capacity to promptly address the comments and resubmit the revised manuscript.

I have thoroughly reviewed the feedback from @thegargiulian and @joaojcorreia . Currently, I am working diligently on revising both the package and the paper. I am hopeful that I will be able to send you the revised version by the end of February.

Thank you for your understanding and patience.

Best regards,

Mashrur

spholmes commented 8 months ago

Dear @mashrur-ayon , Thanks so much for responding promptly so we can move this along, we prefer when papers move even a little at a time, so it is fine to do the revisions in pieces, but we do want to see things moving along, best wishes and good luck in your new endeavours. Susan

spholmes commented 8 months ago

Agreed @arfon , I was waiting for the author to step in and make the changes requested by the reviewers, I'll ping them now to see where they stand on either making the changes or withdrawing the paper.

Hi @arfon, I did get a reply from the author and he has promised to make progress on the revisions. Thanks Susan

arfon commented 8 months ago

Great, thanks for the update all!

mashrur-ayon commented 5 months ago

@editorialbot commands

editorialbot commented 5 months ago

Hello @mashrur-ayon, here are the things you can ask me to do:


# List all available commands
@editorialbot commands

# Get a list of all editors's GitHub handles
@editorialbot list editors

# Adds a checklist for the reviewer using this command
@editorialbot generate my checklist

# Set a value for branch
@editorialbot set joss-paper as branch

# Run checks and provide information on the repository and the paper file
@editorialbot check repository

# Check the references of the paper for missing DOIs
@editorialbot check references

# Generates the pdf paper
@editorialbot generate pdf

# Generates a LaTeX preprint file
@editorialbot generate preprint

# Get a link to the complete list of reviewers
@editorialbot list reviewers
spholmes commented 3 months ago

Hi @mashrur-ayon , I wondered whether you could respond to the revisions requested by the reviewers? If you are not able to respond, how would like to proceed, do you want to withdraw at this time? Thanks, Susan

mashrur-ayon commented 3 months ago

Hi @spholmes

Thank you for your message. I am currently working on the revisions and have almost completed the necessary updates. I will be editing the paper on GitHub and aim to have everything ready by this weekend (7th of July 2024).

Thanks again for your patience and understanding.

Best regards, Mashrur Ayon

mashrur-ayon commented 2 months ago

Dear Reviewers,

@thegargiulian and @joaojcorreia

I hope this message finds you well. I am writing to inform you that I have addressed all the comments and suggestions provided in your feedback for the paper submission on the genderstat package. I appreciate your patience and understanding as I took a bit more time to thoroughly implement the necessary changes.

Summary of Changes:

  1. Clarified Purpose and Data Sources:

    • In the paper and the package I have mentioned that the dataset used was taken from secondary source, and linked the dataset with a DOI, the dataset will be updated on a yearly basis by me.
  2. Simplified Language:

    • Simplified the language across the paper for better readability while maintaining an academic tone.
    • Removed redundant mentions of the package's functionalities to streamline the content.
  3. Detailed Statement of Need:

    • Expanded the statement of need to include theoretical concepts and the necessity of quantitative tools in gender studies.
    • Explained the gap in existing software solutions and how genderstat fills this gap by providing a dedicated tool for gender disparity analysis.
  4. Core Theoretical Concepts:

    • Provided detailed explanations for the Gender Pay Gap (GPG), Gender Inequality Index (GII), Gender Development Index (GDI), and Gender Empowerment Measure (GEM).
    • Discussed the rationale behind choosing these specific metrics.
  5. Application Section:

    • Added a comprehensive application section with case studies showcasing the practical use of the package for analyzing gender disparities.
    • Included code examples and visualizations to illustrate the functionality of the package.
  6. Updated Documentation:

    • Ensured all functions are well-documented with clear examples.
    • Updated the README file on GitHub to reflect the latest changes and included installation instructions for both CRAN and GitHub versions.
  7. Testing and Continuous Integration:

    • Implemented a unit test using the testthat package to ensure the correctness of calculations.
  8. Bibliography and References:

    • Updated the bibliography to include all references in the correct format, ensuring all sources are properly cited.

I sincerely thank you for your valuable feedback, which has significantly improved the quality and robustness of the paper and the genderstat package. Your insights were instrumental in guiding these revisions.

Please find the revised manuscript attached for your review. I look forward to any further feedback you may have and am committed to making any additional improvements as necessary.

Thank you once again for your time and effort.

Best regards,

S M Mashrur Arafin Ayon

mashrur-ayon commented 2 months ago

Dear Editor, @spholmes

I have reviewed and addressed the majority of the comments provided by the reviewers, and have made the necessary changes to the manuscript. I am now generating the updated PDF for submission. If you are satisfied with the revisions, I would be grateful if you could approve my paper. Should you have any further feedback, please let me know, and I will take the necessary steps to ensure the manuscript meets your satisfaction and resubmit as needed.

Thank you for your time, your suggestions and consideration.

Best regards,

S M Mashrur Arafin Ayon

mashrur-ayon commented 2 months ago

@editorialbot generate pdf

editorialbot commented 2 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

mashrur-ayon commented 2 months ago

Kindly accept this updated pdf @editorialbot generate pdf

spholmes commented 2 months ago

@editorialbot check repository

spholmes commented 2 months ago

@editorialbot check references

editorialbot commented 2 months ago

Software report:

github.com/AlDanial/cloc v 1.90  T=0.03 s (1070.8 files/s, 73923.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
CSV                              5              0              0            960
Markdown                         4            140              0            240
XML                             10              0              0            144
R                                5             24            145             87
TeX                              1              5              0             82
YAML                             1              2              0             18
Python                           1              5              1             14
JSON                             1              0              0              1
Rmd                              1             32            102              0
-------------------------------------------------------------------------------
SUM:                            29            208            248           1546
-------------------------------------------------------------------------------

Commit count by author:

    64  Mashrur Arafin Ayon
    58  mashrur ayon
editorialbot commented 2 months ago

Paper file info:

📄 Wordcount for paper.md is 2492

✅ The paper includes a Statement of need section

editorialbot commented 2 months ago

License info:

🔴 Failed to discover a valid open source license

editorialbot commented 2 months ago
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1080/1464988032000125773 is OK
- 10.1257/jel.20160995 is OK
- 10.4324/9781849773560 is OK
- 10.1146/annurev.soc.34.040507.134719 is OK
- 10.4135/9781452275482 is OK
- 10.34740/KAGGLE/DSV/6359326 is OK

MISSING DOIs

- No DOI given, and none found for title: The Simple Truth about the Gender Pay Gap. Fall 20...
- No DOI given, and none found for title: The Gender Inequalities Index (GII) as a new way t...
- No DOI given, and none found for title: R: A Language and Environment for Statistical Comp...
- No DOI given, and none found for title: Human Development Report 2021/2022-Technical Notes

INVALID DOIs

- None
spholmes commented 2 months ago

@mashrur-ayon : Thanks for working on this, while we wait for the reviewers to check your changes, could you check the problem with the open source license. Thanks Susan

mashrur-ayon commented 2 months ago

@mashrur-ayon : Thanks for working on this, while we wait for the reviewers to check your changes, could you check the problem with the open source license. Thanks Susan

@spholmes Unfortunately, I saw that I forgot to add the line License: MIT + file LICENSE in the description file, which I did now. I think the license issue should be solved now.

Thank you very much.

joaojcorreia commented 2 months ago

I will try to look at this over the weekend.

mashrur-ayon commented 2 months ago

I will try to look at this over the weekend.

@joaojcorreia thank you

thegargiulian commented 2 months ago

Hi @spholmes and @mashrur-ayon! Thanks for this update. Unfortunately, I won't be able to re-review the paper and software until mid-August 😥

arfon commented 2 months ago

My apologies to all involved here for triggering this process mid-way through the review (rather than the pre-review issue) but I am concerned about whether this submission meets oursubstantial scholarly effort criterion.

I'm going to trigger a scope review and will return in the coming days with a response from the wider editorial team.

arfon commented 2 months ago

@editorialbot query scope

editorialbot commented 2 months ago

Submission flagged for editorial review.

arfon commented 2 months ago

@mashrur-ayon – after consideration by the JOSS editorial team, this submission has been determined as out of scope for JOSS, failing to meet our substantial scholarly effort criterion.

Again, my apologies to all involved here (especially the reviewers @thegargiulian & @joaojcorreia), this submission should have been flagged at pre-review, and as a result not been sent out for review.

@mashrur-ayon – One possible alternative to JOSS is to follow GitHub's guide on how to create a permanent archive and DOI for your software. This DOI can then be used by others to cite your work.

arfon commented 2 months ago

@editorialbot reject

editorialbot commented 2 months ago

Paper rejected.

mashrur-ayon commented 2 months ago

My apologies to all involved here for triggering this process mid-way through the review (rather than the pre-review issue) but I am concerned about whether this submission meets oursubstantial scholarly effort criterion.

I'm going to trigger a scope review and will return in the coming days with a response from the wider editorial team.

@arfon To address your concerns adequately, could you please clarify which specific aspects of the genderstat package do not meet the substantial scholarly effort criterion?? Understanding the details regarding the commit history, lines of code, number of commits, or any other factors will help me provide the necessary information or make the required adjustments, and resubmit the paper.

arfon commented 2 months ago

JOSS has a desk rejection policy for submissions totalling under 300 lines of code, but in reality, typically anything under ~1000 LOC is flagged for review.

I will also note that the capabilities of this package, while potentially useful, are also very limited which would categorise this as a “Minor utility” package.