search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
696 stars 36 forks source link

[REVIEW]: PySwarming: a research toolkit for Swarm Robotics #5647

Closed editorialbot closed 9 months ago

editorialbot commented 1 year ago

Submitting author: !--author-handle-->@mrsonandrade<!--end-author-handle-- (Emerson Andrade) Repository: https://github.com/mrsonandrade/pyswarming Branch with paper.md (empty if default branch): Version: v1.1.3 Editor: !--editor-->@adi3<!--end-editor-- Reviewers: @sea-bass, @JHartzer Archive: 10.5281/zenodo.8317286

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/24f4cb4bb0dfd65d557497025c351f53"><img src="https://joss.theoj.org/papers/24f4cb4bb0dfd65d557497025c351f53/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/24f4cb4bb0dfd65d557497025c351f53/status.svg)](https://joss.theoj.org/papers/24f4cb4bb0dfd65d557497025c351f53)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@sea-bass & @JHartzer, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @adi3 know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @sea-bass

📝 Checklist for @JHartzer

editorialbot commented 1 year ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 1 year ago 
Software report:

github.com/AlDanial/cloc v 1.88  T=0.03 s (747.4 files/s, 147309.4 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                           7            467            904            945
Jupyter Notebook                 2              0            643            499
Markdown                         3             92              0            285
reStructuredText                 6            126             45            250
TeX                              1             19              0            141
YAML                             2              7              8             47
DOS Batch                        1              8              1             26
make                             1              4              7              9
-------------------------------------------------------------------------------
SUM:                            23            723           1608           2202
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository
editorialbot commented 1 year ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1103/PhysRevLett.75.1226 is OK
- 10.1016/S0921-8890(99)00004-4 is OK
- 10.1109/ICIIS.1999.810278 is OK
- 10.1038/35035023 is OK
- 10.1109/CDC.2001.980728 is OK
- 10.1007/978-3-540-87527-7_36 is OK
- 10.1109/TAC.2003.812781 is OK
- 10.1038/nature03236 is OK
- 10.1017/S0373463308005080 is OK
- 10.3389/frobt.2017.00012 is OK
- 10.1007/s10514-018-9702-0 is OK
- 10.1109/JPROC.2021.3072740 is OK
- 10.3390/s21062062 is OK
- 10.1023/B:AURO.0000033970.96785.f2 is OK
- 10.1016/j.swevo.2019.100565 is OK
- 10.1007/978-3-540-77612-3 is OK
- 10.1109/IROS.2016.7759558 is OK
- 10.1109/LRA.2021.3061366 is OK
- 10.1109/ICRA46639.2022.9812417 is OK
- 10.1007/s11721-012-0075-2 is OK

MISSING DOIs

- None

INVALID DOIs

- None
editorialbot commented 1 year ago

Wordcount for paper.md is 1044

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

adi3 commented 1 year ago

👋🏼 @mrsonandrade this is the review thread for the paper. All of our communications will happen here from now on.

👋🏼 @sea-bass @JHartzer - you both should generate your checklists with the JOSS requirements by running @editorialbot generate my checklist. As you go over the submission, please check the items that you feel have been satisfied and let the author know where further work needs to be done.

Here is a little more context for first-time reviewers :) - The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention https://github.com/openjournals/joss-reviews/issues/5647 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for reviews to be completed within about 2-4 weeks. Please let me know if any of you require some more time. We can also use @editorialbot to set automatic reminders if you know you'll be away for a known period of time.

Please feel free to ping me (@adi3) if you have any questions/concerns. Thank you for all your help!!

sea-bass commented 1 year ago

Review checklist for @sea-bass

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

sea-bass commented 1 year ago

I have gone through and reviewed this submission.

Overall, I think this is a really nice tool and a welcome addition to the swarm robotics community. As someone who has created simular multi-robot simulators (https://github.com/mathworks-robotics/mobile-robotics-simulation-toolbox and https://github.com/sea-bass/pyrobosim), I particularly appreciate that PySwarming isn't just a generic point robot simulator like these tools, but actually is specifically focused to implementing swarm behaviors from the literature in a modular framework.

PySwarming was extremely easy to install (I appreciate that it is pip installable). It was also easy to dig into the Python code and make changes/modifications. Finally, the API documentation and connection to ReadTheDocs was solid.

The code examples were helpful, and I really appreciated that the docs page has a nice Jupyter notebook that renders and shows the animations on the ReadTheDocs page. However, that same notebook didn't run standalone and the main README.md doesn't have self-contained examples beyond the first few simple ones. As such, I highly recommend also including smaller, self-contained examples in plain text .py format in the examples directory. My detailed feedback is here: https://github.com/mrsonandrade/pyswarming/issues/2

The paper is overall in good shape, but I think there are a few structural changes needed. Additionally, some more information on the state of swarm robotics software and on the developer workflows to add their own swarm behaviors to the tool would be helpful. Finally, a conclusion is needed. My detailed feedback is here: https://github.com/mrsonandrade/pyswarming/issues/4

In addition, I found a few more blocking software issues that I think should be resolved:

Thank you for submitting this exciting tool, and I look forward to seeing these comments addressed.

JHartzer commented 1 year ago

Review checklist for @JHartzer

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

sea-bass commented 12 months ago

@editorialbot generate pdf

editorialbot commented 12 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

JHartzer commented 12 months ago

I have a concern regarding substantial scholarly effort of this work. As it currently stands, most of the lines of python code are tests, setups, or examples. While all these things are good to have, what remains are about 400 unique lines of code. Additionally, a large portion of this code is auto_differentiation.py which is used to compute function gradients. Excluding this leaves ~270 lines of code within swarm.py and behaviors.py, which in my opinion does not constitute a substantial scholarly effort.

I can understand that, by nature, swarm robotics has a large variety of objective functions to perform so creating an all-encompassing work is untenable. Other works have focused on specific algorithms such as search and path planning, or are generic but contain high-level examples, like ARGoS. Because this work currently only implements low-level behaviors, I think this currently falls short of the statement of need.

mrsonandrade commented 11 months ago

Dear @JHartzer, I kind of understand your points. However, all these behaviors took me months of work, in which I had to read, understand, and select papers, and put everything in the same nomenclature and format in order to have a common ground. After that, I had to implement while trying to maintain the clearness and connection with their original sources (the articles), all in order to facilitate the understanding for those that are initiating/interested. Also, based on previous experiences, I tried to have good documentation to make it accessible to new researchers. So, please, take this into your scholarly effort considerations.

Regarding the “behaviors”, is exactly what you said, we have “a large variety of objective functions to perform”, but the main point of this library is joining the maximum behaviors/algorithms in order to avoid “repeated” use of those same behaviors/algorithms as something new. For instance, you probably already saw some work that uses a “behavior X” inspired by something, but you knew that the behavior was not new, it was just the same proposed by someone else with a different mathematical expression. So, the main point here is always the algorithms' standardization and implementation in PySwarming (and the use of Python language which needs no words). Also, you mentioned search and path planning, but they have a Python simulator like marabunta, so, PySwarming here is not just another simulator, it comes as a package with various behaviors/algorithms that can be used even within these aforementioned packages. So, I hope you have got my point and I apologize if those points were not clear through the paper.

sea-bass commented 11 months ago

@adi3 After working with the author in response to my comments, I'm inclined to approve on my end. However, @JHartzer makes some valid points in his review. Up to you as the editor on how this should proceed.

@mrsonandrade Thank you for your rebuttal to the comments. While we all appreciate your hard work learning and developing this tool, I unfortunately don't think that the time it took is something that factors in such decisions. Whether it took 1 day or 10 years, the value of these submissions is measured by the submission itself.

JHartzer commented 11 months ago

In addition to the above comments, I have submitted the following blocking issue: https://github.com/mrsonandrade/pyswarming/issues/8

And non-blocking issue: https://github.com/mrsonandrade/pyswarming/issues/7

adi3 commented 11 months ago

@JHartzer @sea-bass @mrsonandrade thanks for all your participation in the review so far! Excellent discussion up there. I'll discuss the reviewers' feedback with other editors and write on how to move forward.

@mrsonandrade in the meantime, could you please address the issues opened by the reviewers? Appreciate your effort!

mrsonandrade commented 11 months ago

Great! Thank you @adi3! I will work on the issues opened by the reviewers.

adi3 commented 11 months ago

@mrsonandrade @JHartzer @sea-bass after taking a deeper look into the codebase and discussing this with other editors, we have concluded that this paper does meet the requirements of substantial scholarly effort. In examinig this, we also took setup, tests and examples into consideration as those are crucial parts of using OSS.

@mrsonandrade please inform @JHartzer when you're done fixing the remaining items so he can complete his review smoothly.

mrsonandrade commented 11 months ago

Thank you @adi3!

Just to inform all of you that two papers related to PySwarming were accepted for publication in the 20th IEEE Latin American Robotics Symposium. As soon as possible, I will insert their information on the README file from PySwarming.

I have worked on the blocking issue from @JHartzer.

Regarding the non-blocking issues from @JHartzer and @sea-bass, I am planning to work on them as soon as possible. I have a new contributor in my lab that will work with this package, so, I hope this will help to work on the non-blocking issues and improve the package.

JHartzer commented 10 months ago

@editorialbot generate pdf

editorialbot commented 10 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

JHartzer commented 10 months ago

The author has addressed my remaining issues and so I can now recommend publication!

adi3 commented 10 months ago

@JHartzer @sea-bass thank you for completing your respective reviews. I appreciate your thoroughness!

@mrsonandrade Looks like everything is set. Congratulations! I need one final formality to be done from your side. Please make a tagged release of the project and archive it. Then report the version number and archive DOI here. Once I have that, I will send this for publication. Cheers!

mrsonandrade commented 10 months ago

@adi3 @sea-bass @JHartzer thank you for all the comments and for your time! 😃😃

@adi3 the version number is: v1.1.3 and the archive DOI is: https://doi.org/10.5281/zenodo.8317286

adi3 commented 9 months ago

@editorialbot set 10.5281/zenodo.8317286 as archive

editorialbot commented 9 months ago

Done! archive is now 10.5281/zenodo.8317286

adi3 commented 9 months ago

@editorialbot set v1.1.3 as version

editorialbot commented 9 months ago

Done! version is now v1.1.3

adi3 commented 9 months ago

@editorialbot generate pdf

adi3 commented 9 months ago

@editorialbot commands

editorialbot commented 9 months ago

Hello @adi3, here are the things you can ask me to do:


# List all available commands
@editorialbot commands

# Add to this issue's reviewers list
@editorialbot add @username as reviewer

# Remove from this issue's reviewers list
@editorialbot remove @username from reviewers

# Get a list of all editors's GitHub handles
@editorialbot list editors

# Assign a user as the editor of this submission
@editorialbot assign @username as editor

# Remove the editor assigned to this submission
@editorialbot remove editor

# Remind an author, a reviewer or the editor to return to a review after a 
# certain period of time (supported units days and weeks)
@editorialbot remind @reviewer in 2 weeks

# Check the references of the paper for missing DOIs
@editorialbot check references

# Perform checks on the repository
@editorialbot check repository

# Adds a checklist for the reviewer using this command
@editorialbot generate my checklist

# Set a value for version
@editorialbot set v1.0.0 as version

# Set a value for branch
@editorialbot set joss-paper as branch

# Set a value for repository
@editorialbot set https://github.com/organization/repo as repository

# Set a value for the archive DOI
@editorialbot set set 10.5281/zenodo.6861996 as archive

# Mention the EiCs for the correct track
@editorialbot ping track-eic

# Generates the pdf paper
@editorialbot generate pdf

# Recommends the submission for acceptance
@editorialbot recommend-accept

# Generates a LaTeX preprint file
@editorialbot generate preprint

# Flag submission with questionable scope
@editorialbot query scope

# Get a link to the complete list of reviewers
@editorialbot list reviewers

# Creates a post-review checklist with editor and authors tasks
@editorialbot create post-review checklist

# Open the review issue
@editorialbot start review
adi3 commented 9 months ago

@editorialbot check references

editorialbot commented 9 months ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1103/PhysRevLett.75.1226 is OK
- 10.1007/s11721-008-0014-4 is OK
- 10.1023/B:AURO.0000033972.50769.1c is OK
- 10.1109/ROBOT.2007.363180 is OK
- 10.1109/IROS.2011.6094829 is OK
- 10.3390/computation10060080 is OK
- 10.1145/37402.37406 is OK
- 10.1103/PhysRevE.58.4828 is OK
- 10.1109/70.736776 is OK
- 10.1016/S0921-8890(99)00004-4 is OK
- 10.1109/ICIIS.1999.810278 is OK
- 10.1038/35035023 is OK
- 10.1109/CDC.2001.980728 is OK
- 10.1007/978-3-540-87527-7_36 is OK
- 10.1109/TAC.2003.812781 is OK
- 10.1038/nature03236 is OK
- 10.1017/S0373463308005080 is OK
- 10.3389/frobt.2017.00012 is OK
- 10.1109/ICRA.2015.7139807 is OK
- 10.1007/s10514-018-9702-0 is OK
- 10.1109/JPROC.2021.3072740 is OK
- 10.3390/s21062062 is OK
- 10.1023/B:AURO.0000033970.96785.f2 is OK
- 10.1007/978-3-540-77612-3 is OK
- 10.1109/IROS.2016.7759558 is OK
- 10.1109/LRA.2021.3061366 is OK
- 10.1109/ICRA46639.2022.9812417 is OK
- 10.1007/s11721-012-0075-2 is OK

MISSING DOIs

- None

INVALID DOIs

- None
editorialbot commented 9 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

adi3 commented 9 months ago

@editorialbot recommend-accept

editorialbot commented 9 months ago 
Attempting dry run of processing paper acceptance...
editorialbot commented 9 months ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1103/PhysRevLett.75.1226 is OK
- 10.1007/s11721-008-0014-4 is OK
- 10.1023/B:AURO.0000033972.50769.1c is OK
- 10.1109/ROBOT.2007.363180 is OK
- 10.1109/IROS.2011.6094829 is OK
- 10.3390/computation10060080 is OK
- 10.1145/37402.37406 is OK
- 10.1103/PhysRevE.58.4828 is OK
- 10.1109/70.736776 is OK
- 10.1016/S0921-8890(99)00004-4 is OK
- 10.1109/ICIIS.1999.810278 is OK
- 10.1038/35035023 is OK
- 10.1109/CDC.2001.980728 is OK
- 10.1007/978-3-540-87527-7_36 is OK
- 10.1109/TAC.2003.812781 is OK
- 10.1038/nature03236 is OK
- 10.1017/S0373463308005080 is OK
- 10.3389/frobt.2017.00012 is OK
- 10.1109/ICRA.2015.7139807 is OK
- 10.1007/s10514-018-9702-0 is OK
- 10.1109/JPROC.2021.3072740 is OK
- 10.3390/s21062062 is OK
- 10.1023/B:AURO.0000033970.96785.f2 is OK
- 10.1007/978-3-540-77612-3 is OK
- 10.1109/IROS.2016.7759558 is OK
- 10.1109/LRA.2021.3061366 is OK
- 10.1109/ICRA46639.2022.9812417 is OK
- 10.1007/s11721-012-0075-2 is OK

MISSING DOIs

- None

INVALID DOIs

- None
editorialbot commented 9 months ago

:wave: @openjournals/pe-eics, this paper is ready to be accepted and published.

Check final proof :point_right::page_facing_up: Download article

If the paper PDF and the deposit XML files look good in https://github.com/openjournals/joss-papers/pull/4603, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

mrsonandrade commented 9 months ago

@adi3 I have checked here and the output looks good 😃

kyleniemeyer commented 9 months ago

Hi @mrsonandrade, I'm doing some final checks before publishing.

I made a few fixes to references in this PR, could you merge it? https://github.com/mrsonandrade/pyswarming/pull/9

mrsonandrade commented 9 months ago

Hi @kyleniemeyer, thank you! done!

kyleniemeyer commented 9 months ago

@editorialbot generate pdf

mrsonandrade commented 9 months ago

@editorialbot check references

editorialbot commented 9 months ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1103/PhysRevLett.75.1226 is OK
- 10.1007/s11721-008-0014-4 is OK
- 10.1023/B:AURO.0000033972.50769.1c is OK
- 10.1109/ROBOT.2007.363180 is OK
- 10.1109/IROS.2011.6094829 is OK
- 10.3390/computation10060080 is OK
- 10.1145/37402.37406 is OK
- 10.1103/PhysRevE.58.4828 is OK
- 10.1109/70.736776 is OK
- 10.1016/S0921-8890(99)00004-4 is OK
- 10.1109/ICIIS.1999.810278 is OK
- 10.1038/35035023 is OK
- 10.1109/CDC.2001.980728 is OK
- 10.1007/978-3-540-87527-7_36 is OK
- 10.1109/TAC.2003.812781 is OK
- 10.1038/nature03236 is OK
- 10.1017/S0373463308005080 is OK
- 10.3389/frobt.2017.00012 is OK
- 10.1109/ICRA.2015.7139807 is OK
- 10.1007/s10514-018-9702-0 is OK
- 10.1109/JPROC.2021.3072740 is OK
- 10.3390/s21062062 is OK
- 10.1023/B:AURO.0000033970.96785.f2 is OK
- 10.1007/978-3-540-77612-3 is OK
- 10.1109/IROS.2016.7759558 is OK
- 10.1109/LRA.2021.3061366 is OK
- 10.1109/ICRA46639.2022.9812417 is OK
- 10.1007/s11721-012-0075-2 is OK

MISSING DOIs

- None

INVALID DOIs

- None
kyleniemeyer commented 9 months ago

@editorialbot generate pdf

editorialbot commented 9 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

kyleniemeyer commented 9 months ago

@editorialbot accept

editorialbot commented 9 months ago 
Doing it live! Attempting automated processing of paper acceptance...
editorialbot commented 9 months ago

Ensure proper citation by uploading a plain text CITATION.cff file to the default branch of your repository.

If using GitHub, a Cite this repository menu will appear in the About section, containing both APA and BibTeX formats. When exported to Zotero using a browser plugin, Zotero will automatically create an entry using the information contained in the .cff file.

You can copy the contents for your CITATION.cff file here:

CITATION.cff

``` cff-version: "1.2.0" authors: - family-names: Andrade given-names: Emerson Martins name-particle: de orcid: "https://orcid.org/0000-0002-5023-8733" - family-names: Fernandes given-names: Antonio Carlos orcid: "https://orcid.org/0000-0001-6578-1985" - family-names: Junior given-names: Joel Sena Sales orcid: "https://orcid.org/0000-0003-4563-1538" doi: 10.5281/zenodo.8317286 message: If you use this software, please cite our article in the Journal of Open Source Software. preferred-citation: authors: - family-names: Andrade given-names: Emerson Martins name-particle: de orcid: "https://orcid.org/0000-0002-5023-8733" - family-names: Fernandes given-names: Antonio Carlos orcid: "https://orcid.org/0000-0001-6578-1985" - family-names: Junior given-names: Joel Sena Sales orcid: "https://orcid.org/0000-0003-4563-1538" date-published: 2023-09-26 doi: 10.21105/joss.05647 issn: 2475-9066 issue: 89 journal: Journal of Open Source Software publisher: name: Open Journals start: 5647 title: "PySwarming: a research toolkit for Swarm Robotics" type: article url: "https://joss.theoj.org/papers/10.21105/joss.05647" volume: 8 title: "PySwarming: a research toolkit for Swarm Robotics" ```

If the repository is not hosted on GitHub, a .cff file can still be uploaded to set your preferred citation. Users will be able to manually copy and paste the citation.

Find more information on .cff files here and here.

editorialbot commented 9 months ago

🐘🐘🐘 👉 Toot for this paper 👈 🐘🐘🐘