search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
704 stars 37 forks source link

[PRE REVIEW]: VoteKit: A Python package for computational social choice research #7088

Open editorialbot opened 1 month ago

editorialbot commented 1 month ago

Submitting author: !--author-handle-->@cdonnay<!--end-author-handle-- (Moon Duchin) Repository: https://github.com/mggg/votekit Branch with paper.md (empty if default branch): JOSS Version: v2.0.1 Editor: !--editor-->@britta-wstnr<!--end-editor-- Reviewers: Pending Managing EiC: Samuel Forbes

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/bb1226313881241bcd98c30e55d4df48"><img src="https://joss.theoj.org/papers/bb1226313881241bcd98c30e55d4df48/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/bb1226313881241bcd98c30e55d4df48/status.svg)](https://joss.theoj.org/papers/bb1226313881241bcd98c30e55d4df48)

Author instructions

Thanks for submitting your paper to JOSS @cdonnay. Currently, there isn't a JOSS editor assigned to your paper.

@cdonnay if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). You can search the list of people that have already agreed to review and may be suitable for this submission.

Editor instructions

The JOSS submission bot @editorialbot is here to help you find and assign reviewers and start the main review. To find out what @editorialbot can do for you type:

@editorialbot commands
editorialbot commented 1 month ago

Hello human, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 1 month ago

Software report:

github.com/AlDanial/cloc v 1.90  T=1.59 s (61.7 files/s, 549181.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JSON                             2              0              0         692798
CSV                             25              7              0         160391
Python                          40           1778           2159           6070
reStructuredText                 9            964           1427           1286
Jupyter Notebook                 5              0           3033            526
Markdown                         4            133              0            468
TeX                              1             65              0            330
YAML                             4              9             28             82
TOML                             2              6              0             44
CSS                              2              4              0             35
DOS Batch                        1              8              1             26
JavaScript                       1              2              0             15
make                             1              4              7              9
Bourne Shell                     1              1              0              2
-------------------------------------------------------------------------------
SUM:                            98           2981           6655         862082
-------------------------------------------------------------------------------

Commit count by author:

   199  Jack Gibson
   174  Chris Donnay
    77  Jennifer Wang
    30  bmacias01
    27  James Turk
    21  gabeschoenbach
    20  ziglaser
    15  Zach Glaser
    10  mduchin
     9  cdonnay
     9  roryhe
     4  gkehne
     4  peterrrock2
     3  Peter
     1  Malavika Mukundan
     1  MalavikaMukundan
editorialbot commented 1 month ago

Paper file info:

📄 Wordcount for paper.md is 2202

✅ The paper includes a Statement of need section

editorialbot commented 1 month ago

License info:

✅ License found: MIT License (Valid open source OSI approved license)

editorialbot commented 1 month ago

:warning: An error happened when generating the pdf. Author (Christopher Donnay) is missing affiliation.

editorialbot commented 1 month ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.21105/joss.04880 is OK
- 10.24963/ijcai.2022/15 is OK
- 10.1257/jep.9.1.27 is OK
- 10.1007/s11127-016-0328-5 is OK
- 10.1007/s00355-011-0603-9 is OK
- 10.2139/ssrn.3778021 is OK
- 10.1023/A:1024221816507 is OK
- 10.1109/ICCV.1998.710701 is OK

MISSING DOIs

- No DOI given, and none found for title: VoteKit
- No DOI given, and none found for title: PrefLib-Tools
- No DOI given, and none found for title: pref_voting
- No DOI given, and none found for title: Votelib
- No DOI given, and none found for title: apportionment
- No DOI given, and none found for title: Scottish STV Election Repo
- No DOI given, and none found for title: COMSOC Community Site
- 10.24963/ijcai.2024/881 may be a valid DOI for title: Guide to Numerical Experiments on Elections in Com...
- No DOI given, and none found for title: RCV Cruncher
- 10.24963/ijcai.2024/881 may be a valid DOI for title: Guide to Numerical Experiments on Elections in Com...
- No DOI given, and none found for title: Drawing a Map of Elections in the Space of Statist...
- No DOI given, and none found for title: Comparing Electoral Systems for the Massachusetts ...
- No DOI given, and none found for title: Analysis of Election Systems for Washington State
- No DOI given, and none found for title: Analysis of Election Systems for the Chelan County...
- No DOI given, and none found for title: Study of Reform Proposals for Chicago City Council
- No DOI given, and none found for title: Findings on the City of Lowell’s Election Systems
- No DOI given, and none found for title: Analysis of Election Systems for Oregon State
- No DOI given, and none found for title: Analysis of Election Systems for the Tukwila, WA S...
- No DOI given, and none found for title: Ranked Choice Voting and Proportional Representati...
- No DOI given, and none found for title: Proportionality for ranked voting, in theory and p...
- No DOI given, and none found for title: Learning blocs and slates from observed elections
- No DOI given, and none found for title: Modeling the Fair Representation Act
- 10.2307/2960485 may be a valid DOI for title: Deterministic and Probabilistic Voting Models
- 10.1007/s42001-024-00249-8 may be a valid DOI for title: A Comparison of Sequential Ranked-Choice Voting an...
- 10.1007/978-94-009-4225-7 may be a valid DOI for title: The Theory of Committees and Elections
- No DOI given, and none found for title: The Source of Election Results: An Empirical Analy...
- No DOI given, and none found for title: The Structure of the Election-Generating Universe
- No DOI given, and none found for title: Electoral system design: The new international IDE...
- 10.1609/aaai.v31i1.10611 may be a valid DOI for title: Proportional justified representation
- 10.1609/aaai.v29i1.9324 may be a valid DOI for title: Justified representation in approval-based committ...
- No DOI given, and none found for title: Proportional rankings
- 10.1609/aaai.v31i1.10612 may be a valid DOI for title: What do multiwinner voting rules do? An experiment...

INVALID DOIs

- None
samhforbes commented 1 month ago

@cdonnay I will begin looking for an editor for this submission, but in the meantime if you don't mind, you can see an error generating the pdf, as well as (less urgently) some missing DOIs in the references.

cdonnay commented 1 month ago

Will do!

There is a similar paper in submission that we think compliments ours very well: https://github.com/openjournals/joss-reviews/issues/7020

Perhaps some of the editors from that would be appropriate for us as well.

mduchin commented 1 month ago

@samhforbes I think we fixed the affiliations, and the DOIs as much as possible. (Many of the references are to white papers that do not have DOIs....) Could you let us know if it looks ok?

samhforbes commented 1 month ago

@editorialbot generate pdf

editorialbot commented 1 month ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

editorialbot commented 1 month ago

Five most similar historical JOSS papers:

PyEI: A Python package for ecological inference Submitting author: @karink520 Handling editor: @drvinceknight (Active) Reviewers: @matt-graham, @pmyteh Similarity score: 0.7171

IDCeMPy: Python Package for Inflated Discrete Choice Models Submitting author: @hknd23 Handling editor: @galessiorob (Active) Reviewers: @cmaimone, @jungtaekkim, @tmickleydoyle Similarity score: 0.6887

electiondata: a Python package for consolidating, checking, analyzing, visualizing and exporting election results Submitting author: @sfsinger19103 Handling editor: @ajstewartlang (Active) Reviewers: @vaneseltine, @andrewheiss Similarity score: 0.6687

Multiple Inference: A Python package for comparing multiple parameters Submitting author: @dsbowen Handling editor: @vissarion (Active) Reviewers: @blakeaw, @mattpitkin, @nhejazi Similarity score: 0.6686

simstudy: Illuminating research methods through data generation Submitting author: @assignUser Handling editor: @mikldk (Retired) Reviewers: @gagolews, @brunaw Similarity score: 0.6540

⚠️ Note to editors: If these papers look like they might be a good match, click through to the review issue for that paper and invite one or more of the authors before considering asking the reviewers of these papers to review again for JOSS.

britta-wstnr commented 3 weeks ago

@editorialbot assign @britta-wstnr as editor

editorialbot commented 3 weeks ago

Assigned! @britta-wstnr is now the editor

britta-wstnr commented 3 weeks ago

Hi @cdonnay 👋 I'll be editing your paper. I'll be back shortly with more information 🙂

britta-wstnr commented 2 weeks ago

@editorialbot check references

britta-wstnr commented 2 weeks ago

@editorialbot generate paper

editorialbot commented 2 weeks ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@editorialbot commands

britta-wstnr commented 2 weeks ago

@editorialbot generate pdf

editorialbot commented 2 weeks ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- 10.21105/joss.04880 is OK
- 10.24963/ijcai.2024/881 is OK
- 10.24963/ijcai.2022/15 is OK
- 10.2139/ssrn.3778021 is OK
- 10.1257/jep.9.1.27 is OK
- 10.2307/2960485 is OK
- 10.1007/s11127-016-0328-5 is OK
- 10.1007/s00355-011-0603-9 is OK
- 10.1007/s42001-024-00249-8 is OK
- 10.1007/978-94-009-4225-7 is OK
- 10.2139/ssrn.3778021 is OK
- 10.1023/A:1024221816507 is OK
- 10.1109/ICCV.1998.710701 is OK
- 10.1609/aaai.v31i1.10611 is OK
- 10.1007/s00355-016-1019-3 is OK
- 10.24963/ijcai.2017/58 is OK
- 10.1609/aaai.v31i1.10612 is OK

🟡 SKIP DOIs

- No DOI given, and none found for title: VoteKit
- No DOI given, and none found for title: PrefLib-Tools
- No DOI given, and none found for title: pref_voting
- No DOI given, and none found for title: Votelib
- No DOI given, and none found for title: apportionment
- No DOI given, and none found for title: Scottish STV Election Repo
- No DOI given, and none found for title: COMSOC Community Site
- No DOI given, and none found for title: RCV Cruncher
- No DOI given, and none found for title: Comparing Electoral Systems for the Massachusetts ...
- No DOI given, and none found for title: Analysis of Election Systems for Washington
 State
- No DOI given, and none found for title: Analysis of Election Systems for the Chelan
 Count...
- No DOI given, and none found for title: Study of Reform Proposals
 for Chicago City Counci...
- No DOI given, and none found for title: Findings on the City of Lowell’s Election
 Systems
- No DOI given, and none found for title: Analysis of Election Systems for Oregon State
- No DOI given, and none found for title: Analysis of Election Systems for the Tukwila,
 WA ...
- No DOI given, and none found for title: Proportionality for ranked voting, in theory and p...
- No DOI given, and none found for title: Learning blocs and slates from observed elections
- No DOI given, and none found for title: Modeling the Fair Representation Act
- No DOI given, and none found for title: The Source of Election Results: An Empirical Analy...
- No DOI given, and none found for title: The Structure of the Election-Generating Universe
- No DOI given, and none found for title: Electoral system design: The new international IDE...

❌ MISSING DOIs

- 10.24963/ijcai.2024/881 may be a valid DOI for title: Guide to Numerical Experiments on Elections in Com...

❌ INVALID DOIs

- 10.5555/3398761.3398916 is INVALID - 10.5555 is a known broken prefix, replace with https://dl.acm.org/doi/{doi} in the {url} field
editorialbot commented 2 weeks ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

editorialbot commented 2 weeks ago

Five most similar historical JOSS papers:

PyEI: A Python package for ecological inference Submitting author: @karink520 Handling editor: @drvinceknight (Active) Reviewers: @matt-graham, @pmyteh Similarity score: 0.7173

IDCeMPy: Python Package for Inflated Discrete Choice Models Submitting author: @hknd23 Handling editor: @galessiorob (Active) Reviewers: @cmaimone, @jungtaekkim, @tmickleydoyle Similarity score: 0.6905

Multiple Inference: A Python package for comparing multiple parameters Submitting author: @dsbowen Handling editor: @vissarion (Active) Reviewers: @blakeaw, @mattpitkin, @nhejazi Similarity score: 0.6718

electiondata: a Python package for consolidating, checking, analyzing, visualizing and exporting election results Submitting author: @sfsinger19103 Handling editor: @ajstewartlang (Active) Reviewers: @vaneseltine, @andrewheiss Similarity score: 0.6661

simstudy: Illuminating research methods through data generation Submitting author: @assignUser Handling editor: @mikldk (Retired) Reviewers: @gagolews, @brunaw Similarity score: 0.6562

⚠️ Note to editors: If these papers look like they might be a good match, click through to the review issue for that paper and invite one or more of the authors before considering asking the reviewers of these papers to review again for JOSS.

britta-wstnr commented 2 weeks ago

Hello @cdonnay 👋 Before inviting reviewers, I have a couple of points for you to address 👷 🙂

  1. You can see that our Editorial Bot now flags a few DOIs as faulty (missing or invalid), see this comment. Can you have a look and see if they're fixable?
  2. Reading your paper, I saw that it is pretty long. Please note that our word limit is around 1000 words. Can you have a look if the paper can be shortened? See here for an overview of what should be included in your paper (although I think you adhere to this pretty well). Maybe you could move some of the information to your documentation? If you have any concerns about this, please let me know!

Thanks a lot! 🙏

britta-wstnr commented 1 week ago

Gentle ping @cdonnay 👋

cdonnay commented 1 week ago

@editorialbot generate pdf

editorialbot commented 1 week ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

editorialbot commented 1 week ago

Five most similar historical JOSS papers:

PyEI: A Python package for ecological inference Submitting author: @karink520 Handling editor: @drvinceknight (Active) Reviewers: @matt-graham, @pmyteh Similarity score: 0.7183

IDCeMPy: Python Package for Inflated Discrete Choice Models Submitting author: @hknd23 Handling editor: @galessiorob (Active) Reviewers: @cmaimone, @jungtaekkim, @tmickleydoyle Similarity score: 0.6897

Multiple Inference: A Python package for comparing multiple parameters Submitting author: @dsbowen Handling editor: @vissarion (Active) Reviewers: @blakeaw, @mattpitkin, @nhejazi Similarity score: 0.6708

electiondata: a Python package for consolidating, checking, analyzing, visualizing and exporting election results Submitting author: @sfsinger19103 Handling editor: @ajstewartlang (Active) Reviewers: @vaneseltine, @andrewheiss Similarity score: 0.6704

simstudy: Illuminating research methods through data generation Submitting author: @assignUser Handling editor: @mikldk (Retired) Reviewers: @gagolews, @brunaw Similarity score: 0.6542

⚠️ Note to editors: If these papers look like they might be a good match, click through to the review issue for that paper and invite one or more of the authors before considering asking the reviewers of these papers to review again for JOSS.

cdonnay commented 1 week ago

Hi @britta-wstnr.

For length, we are thinking of cutting the last two sections: 1) Area of need: resources for research and 2) projects. Do you think that would be appropriate?

We have edited the suggested doi and the invalid doi, but the rest of the articles have no DOI.

britta-wstnr commented 5 days ago

Hi @cdonnay,

from my point of view, mentioning projects that use your software is quite useful, it's also in our paper guidelines. Maybe you can list them a little more concisely. I think you could save some text by making some of your examples (both theoretic and code/figure examples) a little less verbose. You could for example consider moving the details to your documentation and referring there. But I am also curious what @samhforbes thinks about this!

Just as a note because I happened to come across this right now: in the last paper link you generated above, some references appear as ?, e.g. in footnote 4 on page 2 and in line 119.

samhforbes commented 5 days ago

@editorialbot generate pdf

editorialbot commented 5 days ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

editorialbot commented 5 days ago

Five most similar historical JOSS papers:

PyEI: A Python package for ecological inference Submitting author: @karink520 Handling editor: @drvinceknight (Active) Reviewers: @matt-graham, @pmyteh Similarity score: 0.7172

IDCeMPy: Python Package for Inflated Discrete Choice Models Submitting author: @hknd23 Handling editor: @galessiorob (Active) Reviewers: @cmaimone, @jungtaekkim, @tmickleydoyle Similarity score: 0.6884

Multiple Inference: A Python package for comparing multiple parameters Submitting author: @dsbowen Handling editor: @vissarion (Active) Reviewers: @blakeaw, @mattpitkin, @nhejazi Similarity score: 0.6698

electiondata: a Python package for consolidating, checking, analyzing, visualizing and exporting election results Submitting author: @sfsinger19103 Handling editor: @ajstewartlang (Active) Reviewers: @vaneseltine, @andrewheiss Similarity score: 0.6684

simstudy: Illuminating research methods through data generation Submitting author: @assignUser Handling editor: @mikldk (Retired) Reviewers: @gagolews, @brunaw Similarity score: 0.6534

⚠️ Note to editors: If these papers look like they might be a good match, click through to the review issue for that paper and invite one or more of the authors before considering asking the reviewers of these papers to review again for JOSS.

samhforbes commented 5 days ago

Hi @cdonnay I'd agree with @britta-wstnr here - I think it's useful and even recommended to have some examples of current use. It's obviously your paper, but if it were me I'd look to economise on each of the areas of need sections in particular. I think they could certainly go somewhere in software documentation, but the details you provide in this paper don't need to be full ones.

cdonnay commented 1 day ago

Great we'll give that a go and post here when we feel it's been shortened.