[PRE REVIEW]: FedIRT: An R package and shiny app for estimating federated item response theory models

[editorialbot]

Submitting author: !--author-handle-->@BiyingZhou<!--end-author-handle-- (Biying Zhou) Repository: https://github.com/Feng-Ji-Lab/FedIRT Branch with paper.md (empty if default branch): main Version: v0.1.0 Editor: Pending Reviewers: Pending Managing EiC: Olivia Guest

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/554b550f5d5bf348d76dab9b013df66a"><img src="https://joss.theoj.org/papers/554b550f5d5bf348d76dab9b013df66a/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/554b550f5d5bf348d76dab9b013df66a/status.svg)](https://joss.theoj.org/papers/554b550f5d5bf348d76dab9b013df66a)

Author instructions

Thanks for submitting your paper to JOSS @BiyingZhou. Currently, there isn't a JOSS editor assigned to your paper.

@BiyingZhou if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). You can search the list of people that have already agreed to review and may be suitable for this submission.

Editor instructions

The JOSS submission bot @editorialbot is here to help you find and assign reviewers and start the main review. To find out what @editorialbot can do for you type:

@editorialbot commands

[editorialbot]

Hello human, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.90  T=0.02 s (1539.6 files/s, 203676.9 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
R                               22            275            686           2176
XML                              1              0              0            520
CSV                              5              0              0            501
Markdown                         3             70              0            112
TeX                              1              5              0             81
YAML                             2             11              6             55
-------------------------------------------------------------------------------
SUM:                            34            361            692           3445
-------------------------------------------------------------------------------

Commit count by author:

    14  BiyingZhou
     2  trtfj
     1  Biying Zhou

[editorialbot]

Paper file info:

📄 Wordcount for paper.md is 1285

✅ The paper includes a Statement of need section

[editorialbot]

License info:

🟡 License found: Other (Check here for OSI approval)

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- None

MISSING DOIs

- 10.4324/9781138609877-ree61-1 may be a valid DOI for title: Item response theory
- No DOI given, and none found for title: Communication-efficient learning of deep networks ...
- No DOI given, and none found for title: Federated Psychometrics: A Distributed, Privacy-pr...
- No DOI given, and none found for title: Federated Item Response Theory: A Distributed, Pri...
- No DOI given, and none found for title: Federated Item Response Models
- No DOI given, and none found for title: mirt: A multidimensional item response theory pack...
- No DOI given, and none found for title: ltm: An R package for latent variable modeling and...
- No DOI given, and none found for title: Predictive Modeling of Uniform Differential Item F...
- No DOI given, and none found for title: Federated learning: Strategies for improving commu...
- No DOI given, and none found for title: Multilevel IRT modeling in practice with the packa...
- 10.1007/bf02293801 may be a valid DOI for title: Marginal maximum likelihood estimation of item par...

INVALID DOIs

- None

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[oliviaguest]

@BiyingZhou can you check the DOIs, please? ☺️

[oliviaguest]

License info:

🟡 License found: Other (Check here for OSI approval)

I see: https://github.com/Feng-Ji-Lab/FedIRT/blob/main/LICENSE and https://github.com/Feng-Ji-Lab/FedIRT/blob/main/LICENSE/md — maybe they can be merged/tidied up into one plain text file called LICENSE with content like this: https://github.com/git/git-scm.com/blob/main/MIT-LICENSE.txt?

[BiyingZhou]

@oliviaguest Thank you for your comments. The DOIs for references are now added and the LICENSE is modified as you suggested.

[BiyingZhou]

@editorialbot commands

[editorialbot]

Hello @BiyingZhou, here are the things you can ask me to do:

# List all available commands
@editorialbot commands

# Get a list of all editors's GitHub handles
@editorialbot list editors

# Adds a checklist for the reviewer using this command
@editorialbot generate my checklist

# Set a value for branch
@editorialbot set joss-paper as branch

# Run checks and provide information on the repository and the paper file
@editorialbot check repository

# Check the references of the paper for missing DOIs
@editorialbot check references

# Generates the pdf paper
@editorialbot generate pdf

# Generates a LaTeX preprint file
@editorialbot generate preprint

# Get a link to the complete list of reviewers
@editorialbot list reviewers

[BiyingZhou]

@editorialbot check repository

[BiyingZhou]

@editorialbot check references

[BiyingZhou]

@editorialbot generate pdf

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.90  T=0.02 s (1528.8 files/s, 207505.3 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
R                               22            275            686           2176
XML                              1              0              0            520
CSV                              5              0              0            501
Markdown                         2             66              0             95
TeX                              1              1              0             87
YAML                             2             11              6             55
-------------------------------------------------------------------------------
SUM:                            33            353            692           3434
-------------------------------------------------------------------------------

Commit count by author:

    16  BiyingZhou
     5  Biying Zhou
     2  trtfj

[editorialbot]

Paper file info:

📄 Wordcount for paper.md is 1285

✅ The paper includes a Statement of need section

[editorialbot]

License info:

✅ License found: MIT License (Valid open source OSI approved license)

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.4324/9781410605269 is OK
- 10.18637/jss.v048.i06 is OK
- 10.18637/jss.v017.i05 is OK
- 10.48550/arXiv.1610.05492 is OK
- 10.18637/jss.v020.i05 is OK
- 10.1007/BF02293801 is OK

MISSING DOIs

- No DOI given, and none found for title: Communication-efficient learning of deep networks ...
- No DOI given, and none found for title: Federated Psychometrics: A Distributed, Privacy-pr...
- No DOI given, and none found for title: Federated Item Response Theory: A Distributed, Pri...
- No DOI given, and none found for title: Federated Item Response Models
- No DOI given, and none found for title: Predictive Modeling of Uniform Differential Item F...

INVALID DOIs

- None

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[BiyingZhou]

@editorialbot generate pdf

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[BiyingZhou]

Submitting author: @BiyingZhou (Biying Zhou) Repository: https://github.com/Feng-Ji-Lab/FedIRT Branch with paper.md (empty if default branch): main Version: v0.1.0 Editor: Pending Reviewers: Pending Managing EiC: Olivia Guest

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/554b550f5d5bf348d76dab9b013df66a"><img src="https://joss.theoj.org/papers/554b550f5d5bf348d76dab9b013df66a/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/554b550f5d5bf348d76dab9b013df66a/status.svg)](https://joss.theoj.org/papers/554b550f5d5bf348d76dab9b013df66a)

Author instructions

Thanks for submitting your paper to JOSS @BiyingZhou. Currently, there isn't a JOSS editor assigned to your paper.

@BiyingZhou if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). You can search the list of people that have already agreed to review and may be suitable for this submission.

Editor instructions

The JOSS submission bot @editorialbot is here to help you find and assign reviewers and start the main review. To find out what @editorialbot can do for you type:

@editorialbot commands

Hello @oliviaguest , I suggest these reviewers: ssriv, Ziqian-xia, and JeffreyCHoover. Would you like to take some time to assign reviewers for my paper? Thank you!

[oliviaguest]

@BiyingZhou thank you so much. I need to find you an editor first, but this is much appreciated! ☺️

[oliviaguest]

@pdebuyl @vissarion @fboehm @HaoZeke @spholmes @lrnv do any of you have capacity to edit this coming up?

[HaoZeke]

@oliviaguest, I think I could, however, after a quick look at the repo, perhaps this ought to go through the scope query process?

My concern is that the majority of the fedirt_* files mention This function is only used to test the accuracy and processing time of this algorithm. which means, the software report by linguist is a little inflated. Also those functions are all rather similar. It seems that the main code is fedirt_gpcm.R with ~300 lines..

[oliviaguest]

@editorialbot query scope

[editorialbot]

Submission flagged for editorial review.

[oliviaguest]

@oliviaguest, I think I could, however, after a quick look at the repo, perhaps this ought to go through the scope query process?

My concern is that the majority of the fedirt_* files mention This function is only used to test the accuracy and processing time of this algorithm. which means, the software report by linguist is a little inflated. Also those functions are all rather similar. It seems that the main code is fedirt_gpcm.R with ~300 lines..

Oh, thanks! I'm not an R expert and this sounds sensible. Please do place your concerns in the internal process comment box.

[BiyingZhou]

@editorialbot generate pdf

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[arfon]

@editorialbot reject

@BiyingZhou - thanks for your submission to JOSS. Unfortunately, after review by the JOSS editorial team we've determined that this submission doesn't meet our substantial scholarly effort criterion.

One possible alternative to JOSS is to follow GitHub's guide on how to create a permanent archive and DOI for your software. This DOI can then be used by others to cite your work.

[editorialbot]

Paper rejected.