search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
694 stars 36 forks source link

[REVIEW]: EdgeVPN.io: Seamless Software-defined Layer 2 Virtual Networking for Edge Computing #6638

Open editorialbot opened 2 months ago

editorialbot commented 2 months ago

Submitting author: !--author-handle-->@kcratie<!--end-author-handle-- (Kensworth Subratie) Repository: https://github.com/EdgeVPNio/evio Branch with paper.md (empty if default branch): joss Version: v24.1.2.1061 Editor: !--editor-->@sneakers-the-rat<!--end-editor-- Reviewers: @abhishektiwari, @pradeeban Archive: Pending

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/620c2eca789349609e694fe8dc26988b"><img src="https://joss.theoj.org/papers/620c2eca789349609e694fe8dc26988b/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/620c2eca789349609e694fe8dc26988b/status.svg)](https://joss.theoj.org/papers/620c2eca789349609e694fe8dc26988b)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@abhishektiwari & @pradeeban, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @sneakers-the-rat know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @abhishektiwari

📝 Checklist for @pradeeban

editorialbot commented 2 months ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 2 months ago

Software report:

github.com/AlDanial/cloc v 1.90  T=0.08 s (480.0 files/s, 162318.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          31           1121           1399           9945
TeX                              1              8              0            115
JSON                             2              0              0             94
Markdown                         3             35              0             72
Dockerfile                       1              4             17             39
-------------------------------------------------------------------------------
SUM:                            38           1168           1416          10265
-------------------------------------------------------------------------------

Commit count by author:

   124  Ken
    49  Ken Subratie
     4  Rajath Ganesh
     4  Vahid Daneshmand
     4  ken
     3  Prajwala
     1  Renato Figueiredo
     1  Saumitra Aditya
editorialbot commented 2 months ago

Paper file info:

📄 Wordcount for paper.md is 923

✅ The paper includes a Statement of need section

editorialbot commented 2 months ago

License info:

✅ License found: MIT License (Valid open source OSI approved license)

editorialbot commented 2 months ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5281/zenodo.10655929 is OK
- 10.1016/j.future.2022.10.007 is OK
- 10.17487/RFC3920 is OK
- 10.1145/1355734.1355746 is OK
- 10.17487/RFC7047 is OK
- 10.17487/RFC8926 is OK
- 10.17487/RFC5245 is OK

MISSING DOIs

- No DOI given, and none found for title: Symphony: Distributed Hashing in a Small World
- No DOI given, and none found for title: The cloud is not enough: Saving {IoT} from the clo...

INVALID DOIs

- None
editorialbot commented 2 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

sneakers-the-rat commented 2 months ago

Hello @abhishektiwari and @pradeeban, thank you for agreeing to review this package!

As the top issue says, you'll be working from a checklist, a set of guidelines, and some review criteria.

Goals of Review

The goal here is to collaboratively help this package reach a basic standard for usability, maintainability, and correctness - we are all friends here, there are no gates to be kept, we're just trying to learn from and help each other out. Feel free to talk amongst each other as peers in the issues related to this review. This is an open and iterative review, so you don't need to wait until you've reviewed everything before raising an issue or commenting - in fact its preferable to raise a comment early so that the authors have time to respond and make any requested changes while you're completing the rest of your review.

The checklist is a set of minimum standards that the package has to meet, but you are also free to read, comment, raise issues and pull requests, and do whatever else falls under your purview as a reviewer. Think about what you would want to have if you were a potential new user of the package - is the documentation clear? do the tests cover what they need to? etc. That kind of neutral third-party perspective is invaluable for making software accessible and maintainable.

Process

As you review, please

Since github issues don't have threading, this is how we can keep discussion organized, because commenting in this issue can be a bit messy. Don't be shy commenting here to ask questions about the review itself, though!

It's up to you all and the maintainers if you want to follow any sort of naming convention for issues/PRs - authors please comment here about any preferences you have for raised issues. I would recommend at least tagging issues by general topic areas - [docs], [tests], [bug], etc. and also if the reviewers would indicate if a review is a [question], [suggestion] or a [blocker] for the review so the authors know if a given issue needs some fix in order for the review to be completed.

Other than that, i'll be here so feel free to ask questions! The first thing you'll want to do is generate your checklists using the commands in the OP comment

abhishektiwari commented 2 months ago

Review checklist for @abhishektiwari

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

pradeeban commented 2 months ago

Review checklist for @pradeeban

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

abhishektiwari commented 2 months ago

@kcratie Please see my initial comments,

  1. Can you clarify what repositories under https://github.com/EdgeVPNio are under the scope for this paper? Repository linked to this submission is evio. Paper itself mentions two repositories evio and tincan, but there are other complimentary repositories like portal, tools, etc.

  2. Also can you provide temporary XMPP/TURN/STUN accounts to reviewers to help us bootstrap Evio overlay? I would avoid spending my time and money to install and run XMPP/TURN/STUN on AWS just to verify the software claims.

Tagging @sneakers-the-rat for visibility.

sneakers-the-rat commented 2 months ago

Yes correct @abhishektiwari it is the responsibility of the authors to make sure you have what you need to complete the review - @kcratie any guidance on setting up a development/testing environment?

renatof commented 1 month ago

Thanks, folks! Regarding temporary XMPP/TURN accounts, we had provided a zip file as part of the submission, but you can also request trial accounts using the form linked from here: https://edgevpn.io/trial/

abhishektiwari commented 1 month ago

Thanks @renatof

Did you say zip file? Where can I find it?

abhishektiwari commented 1 month ago

Can you please also confirm repositories covered by the paper?

renatof commented 1 month ago

@kcratie Please see my initial comments,

  1. Can you clarify what repositories under https://github.com/EdgeVPNio are under the scope for this paper? Repository linked to this submission is evio. Paper itself mentions two repositories evio and tincan, but there are other complimentary repositories like portal, tools, etc.
  2. Also can you provide temporary XMPP/TURN/STUN accounts to reviewers to help us bootstrap Evio overlay? I would avoid spending my time and money to install and run XMPP/TURN/STUN on AWS just to verify the software claims.

Tagging @sneakers-the-rat for visibility.

Folks, our answers below (tagging @abhishektiwari @pradeeban)

  1. confirming that the only repositories under the scope of the paper are evio and tincan
  2. attaching a zip file here JOS1.zip with 5 configuration files for trial accounts that can be used for testing as per requested. Instructions for running Evio with Docker using these accounts are available here
sneakers-the-rat commented 1 month ago

Thank you @renatof - reviewers please let me know if there are still difficulties. If there is some lack of clarity in the documentation that the reviewers feel like would hinder normal use, they should feel free to describe what difficulties they had and make suggestions to improve onboarding/make it more obvious how to use the tool. Thats part of the goal of the review! To give an external perspective on things that might be hard for people close to the project to see as potential barriers.

sneakers-the-rat commented 1 month ago

Checking in - do the reviewers need anything from me or the authors? no rush :)

pradeeban commented 1 month ago

Unfortunately, I do not have two Linux computers (x86/amd64 or arm64-based) running Ubuntu 20.04 or later: A requirement to test this product, as per the documentation.

Testing the functionality of this product appears to be an involved process, and I am afraid I will not be able to verify the below two aspects satisfactorily. I have confirmed that everything else is perfectly in order, though. I would like to end my review with this note so that you will be able to find another reviewer who can indeed confirm the functionality on the below aspects:

sneakers-the-rat commented 1 month ago

Thank you for letting us know. yes having two Linux PCs is a bit more than the typical expectations of a reviewer, let me check with the EIC re: how to handle this and I'll get back to you

renatof commented 1 month ago

Thanks for the updates, folks - requiring two computers is indeed a bit more than the typical expectation, and while an ideal scenario for testing would be two separate computers, another possibility is to test on two VMs on the same host.