Closed editorialbot closed 6 months ago
Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.
For a list of things I can do to help you, just type:
@editorialbot commands
For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:
@editorialbot generate pdf
Software report:
github.com/AlDanial/cloc v 1.88 T=0.53 s (189.1 files/s, 428840.5 lines/s)
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
Python 48 2280 5245 5609
Jupyter Notebook 12 0 211574 1108
reStructuredText 21 389 358 818
Markdown 6 204 0 554
DOS Batch 1 34 2 227
TeX 1 23 0 206
YAML 6 37 29 178
make 2 34 22 67
INI 2 4 2 35
TOML 2 0 0 27
-------------------------------------------------------------------------------
SUM: 101 3005 217232 8829
-------------------------------------------------------------------------------
gitinspector failed to run statistical information for the repository
Wordcount for paper.md
is 1206
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):
OK DOIs
- 10.1007/978-3-030-48478-1_1 is OK
- 10.3390/math10050759 is OK
- 10.1109/hipc53243.2021.00045 is OK
- 10.1109/tvcg.2022.3153895 is OK
- 10.1140/epjds/s13688-020-00231-0 is OK
- 10.1371/journal.pcbi.1011190 is OK
- 10.1016/s0924-6509(09)x7013-3 is OK
- 10.1016/s0924-6509(08)x7007-2 is OK
- 10.1186/s12859-021-04197-2 is OK
- 10.48550/arXiv.2208.03103 is OK
- 10.1093/comnet/cnad019 is OK
- 10.48550/arXiv.2208.06894 is OK
- 10.1007/978-3-030-67318-5_25 is OK
- 10.1137/20M1355896 is OK
- 10.1007/978-3-030-71704-9_1 is OK
- 10.21105/joss.05162 is OK
- 10.1007/978-3-030-25070-6_9 is OK
- 10.1101/2023.09.29.560191 is OK
MISSING DOIs
- None
INVALID DOIs
- None
:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:
@szhorvat, @IvanIsCoding, and @drj11 - Thanks for agreeing to review this submission. This is the review thread for the paper. All of our communications will happen here from now on.
As you can see above, you each should use the command @editorialbot generate my checklist
to create your review checklist. @editorialbot commands need to be the first thing in a new comment.
As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.
The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention openjournals/joss-reviews#6016
so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.
We aim for reviews to be completed within about 4 weeks. Please let me know if either of you require some more time. We can also use editorialbot (our bot) to set automatic reminders if you know you'll be away for a known period of time.
Please feel free to ping me (@danielskatz) if you have any questions/concerns.
@editorialbot add @drj11 as reviewer
@drj11 added to the reviewers list!
I have reviewed the software package and preliminarily I believe it would be a great addition to the papers published at the Journal.
With regards to the software, I do not have objections. I believe HyperNetX is a significant scholarly contribution and that it is as mature as if not more mature than other packages published in this journal.
From a Software Engineering point of view, I also found the that the test suite of the package meets the standards for publication. I was able to install the software and run the tests without issues. I was also able to generate a code coverage report. There are some warnings when running the tests and the coverage for the Drawing module is on the lower side. But overall the software is tested and Continuous Integration is in place to detect regressions, which is nice.
For the documentation I focused more on the provided Jupyter Notebooks and A Gentle Introduction to Hypergraph Mathematics provided in the docs. There are also documentations for every function and class, but I admit I haven't scrutinized each method and looked at the greater picture.
As a reviewer with a greater emphasis on graphs than hypegraphs, I found that the "gentle" introduction answered most of the questions for the statement of need and defined the terminology for the package.
The Jupyter Notebooks available in Colab were also a pleasant surprise. I think they exemplify how to use the package and highlight the visualization features that come with HyperNetX.
This is the only section where I think there is feedback to give to the authors from my side. Namely:
pandas
following the guidelines of the packageNetworkX
following the guidelines of the package👋 @brendapraggastis - while two reviews are currently in progress, you could respond to (or act on) @IvanIsCoding's comments and issue during this time.
Thanks @IvanIsCoding and @danielskatz . We are addressing your comments and should be able to push paper changes in the next week or so.
Sorry about the long absence. If all goes according to plan I hope to finish the review by Dec 14.
@IvanIsCoding Thanks for the detailed review. I've incorporated your suggesttions in the paper in https://github.com/pnnl/HyperNetX/tree/paper/HNX_Paper.
@editorialbot generate pdf
@brendapraggastis - you could issue this command too, if you make further changes. I'll also check the references again next. Commands need to be the first thing in a comment
@editorialbot check references
:warning: An error happened when generating the pdf. Problem with ORCID (0000-0000-0000-0000) for Sinan Aksoy. Invalid ORCID.
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):
OK DOIs
- 10.1007/978-3-030-48478-1_1 is OK
- 10.3390/math10050759 is OK
- 10.1109/hipc53243.2021.00045 is OK
- 10.1109/tvcg.2022.3153895 is OK
- 10.1140/epjds/s13688-020-00231-0 is OK
- 10.1371/journal.pcbi.1011190 is OK
- 10.1016/s0924-6509(09)x7013-3 is OK
- 10.1016/s0924-6509(08)x7007-2 is OK
- 10.1186/s12859-021-04197-2 is OK
- 10.48550/arXiv.2208.03103 is OK
- 10.1093/comnet/cnad019 is OK
- 10.48550/arXiv.2208.06894 is OK
- 10.1007/978-3-030-67318-5_25 is OK
- 10.1137/20M1355896 is OK
- 10.1007/978-3-030-71704-9_1 is OK
- 10.21105/joss.05162 is OK
- 10.1007/978-3-030-25070-6_9 is OK
- 10.1101/2023.09.29.560191 is OK
- 10.5281/zenodo.3509134 is OK
- 10.25080/Majora-92bf1922-00a is OK
MISSING DOIs
- None
INVALID DOIs
- None
@brendapraggastis - if an author has an ORCID, please enter it. If not, please remove the line about it.
@editorialbot generate pdf
:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:
@danielskatz I put the ORCID numbers back in. I believe it is up-to-date now.
👋 @szhorvat and @drj11 - I'm checking in to see how your reviews are coming, and if there's anything blocking your progress I can help with.
On it! Thanks for the nudge.
I have completed my review. While i have not poked into every corner of functionality described in the paper, i have poked around enough to be satisfied that the functionality of HyperNetX is substantially as described in its documentation. I find no further reason to delay publication, and recommend that this paper be published by JOSS.
I find the subject to be quite novel and interesting (particularly coming from a more trad graph theory background), and i liked the inclusion of Les Misérables in the tutorials. The package as a whole is made to a high standard, and integrates well with the established Python ecosystems. It is a credit to both the Python and the scientific programming community. I commend the authors for their efforts.
My only real substantial comment, and it is relatively minor, is that i would spent some time on your very long pylint
and my own particular bugbear: bare-except
.
thanks @drj11!
👋 @szhorvat - how is your review coming?
Thanks for the reminder @danielskatz. I was travelling in China for 3 weeks, and accessing GitHub, or even PyPI, turned out to be much less convenient than I expected due to the GFW. I got home on Monday and back to work now.
👋 @szhorvat - I'm just checking in post-holidays on how your review is going. I look forward to seeing progress, or some discussion about what's blocking you. (Given that we have two complete reviews, we could move forward, but since you started on this, I want to give you the chance to finish and add your contributions to the process, assuming this is still what you want to do.)
@IvanIsCoding I have addressed the issues that your raised and merged in the changes in both the paper
and master
branch; the changes can be referenced via the commit message. Any commit message prefixed with HYP-364
refers to the requested changes for this review. @szhorvat The paper
branch has been updated and ready for review. Please let me know if you need anything else or have any questions.
@editorialbot generate pdf
Note to @bonicim - you could also run this command if there are further changes
:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:
👋 @IvanIsCoding & @drj11 - it looks like you've now checked off all the items on your reviews. Can you confirm that you are ok with this being published now?
👋 @szhorvat - How is your review coming? When do you think it might be complete?
@danielskatz my review is (still) complete, and i am okay with proceeding to publish.
I am working on it at the moment and have the full day tomorrow set aside for this. So far it looks like a very solid package.
@brendapraggastis When citing igraph in the paper, could you please spell it with a lowercase "i" and also include https://arxiv.org/abs/2311.10260 (which will eventually become the main citation point, replacing the old DOI-less paper)?
@szhorvat Thanks for the careful read. We've assigned most of your issues to be addressed either in a hotfix to the documentation (as in the error messages for empty hypergraphs, Issue #137 ) or a future release. Are there any specific issues you see that you believe must be remedied before publishing in JOSS?
@brendapraggastis Let's set the end of the week (Sunday) as the time point for finalizing the review. I'm sorry that this took so long. I am still playing with the library and will open issues until then.
Could you please answer here? https://github.com/pnnl/HyperNetX/issues/135 This is not something for you to fix. I just need to know the best channels to ask for help.
You should treat all issues I open as suggestions for now.
Something that I do believe should be fixed in the paper (not library) is a better summary of related and alternative software. At the moment you are simply referencing a different paper that is: (1) over two years old in a rapidly developing area (2) is not actually a review paper on the topic (3) is paywalled and I cannot access it (can you email it to me please?) I would expect not just a list of alternatives, but some sort of useful guidance for users on what packages to look at given their application area, and how HyperNetX differs from these. I understand that JOSS uses a very constraining word limit, but you are not at the word limit, and if the paper is to be useful to readers, this is an important area to improve. Looking at how other related software handle this, XGI has a list of related/alternative packages right in their README: https://github.com/xgi-org/xgi
@szhorvat I have answered https://github.com/pnnl/HyperNetX/issues/135. You can now start a general discussion here.
Before I can check off this item:
- Contribution and authorship: Has the submitting author (@brendapraggastis) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
I wanted to note that only 3 out of the 8 authors seem to have made commits to the repository, https://github.com/pnnl/HyperNetX/graphs/contributors (unless I failed to match up some GitHub user names). Authorship policy is up to the journal (and personally I certainly do not think that authorship should require code contributions). Therefore I leave it up to the editors whether they need further clarifications on this.
Before I can check off this item:
- Contribution and authorship: Has the submitting author (@brendapraggastis) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
I wanted to note that only 3 out of the 8 authors seem to have made commits to the repository, https://github.com/pnnl/HyperNetX/graphs/contributors (unless I failed to match up some GitHub user names). Authorship policy is up to the journal (and personally I certainly do not think that authorship should require code contributions). Therefore I leave it up to the editors whether they need further clarifications on this.
@szhorvat All listed authors have contributed to the repository. Ongoing code adjustments and documentation must be reviewed by our company before public release so the true commit history isn't visible on Github.
It looks like there is some discussion/work going on between @brendapraggastis @bonicim and @szhorvat, so I think we're waiting for these issues to get resolved before we can proceed. If anyone disagrees, please say so.
@danielskatz and @szhorvat - it looks like the only thing pending is a better description of other hypergraph libraries. This will take some time depending on how in depth a comparison we make. @szhorvat suggests "I would expect not just a list of alternatives, but some sort of useful guidance for users on what packages to look at given their application area, and how HyperNetX differs from these." I want to make sure we fairly represent the other libraries and a deep dive into their inner workings will take time. I will discuss this with my team and follow up with you later this week.
@brendapraggastis - thanks for your update. However, in addition to what you say, I also see a 11 open issues that are tagged with this review, I think all from @szhorvat. It's unclear to me how many of these are blockers for @szhorvat (vs suggestions for the future), but given the unchecked review items in @szhorvat's checklist, I assume some of them do need to be addressed. @szhorvat - perhaps you can make it clear which of these issues fall into which category?
I would like to suggest fixing the following before publication:
I want to make sure we fairly represent the other libraries and a deep dive into their inner workings will take time.
To be clear, I am not asking for a deep dive, after all this is not a review paper. However, currently there is no guidance at all for readers about what alternatives exist and what makes HNX different. There is a figure (Fig 2) with no main text reference, and a citation to Kurte et al, which I still wasn't able to access.
With that out of the way, I wanted to say that HNX looks like a great package that is breaking new ground. Modelling systems using higher order networks is still a relatively new, and quickly developing approach. I enjoyed exploring HNX and I'm looking forward to seeing significant development in the coming years.
@szhorvat https://rdcu.be/dw897 I was able to share the Kurte et al. paper. I hope that helps
- [joss] Testing instructions are incomplete in the documentation pnnl/HyperNetX#145 (quick and easy doc fix)
Will fix.
- [joss] Contributor guide should be more specific, and probably shouldn't suggest Rust as the language pnnl/HyperNetX#144 I consider the contributor guide important for any open source project.
Will fix.
- [joss] HyperNetX Widget not working pnnl/HyperNetX#143 : This can't be fixed quickly, and it wouldn't be reasonable to ask for this to be fixed before publishing. But given that this feature is prominently mentioned in the paper, and currently it is quite difficult to get it working, could the authors please comment on what the plans are going forward? Will it be supported and made easy to use, or will it be kept in its current state or dropped?
The HyperNetX widget is currently in beta with limitations on the Jupyter environment in which it may be used. It is being actively worked on. Look for improvements and an expanded list of usable environments in a future release.
In addition, updates to the documentation are currently in progress and are planned to be published in the next week or so.
- [joss] More clarity on (optional) dependencies pnnl/HyperNetX#142 This is not a major issue, but again it is a quick fix, and I'd like to see it corrected for the sake of user friendliness.
Will fix.
Submitting author: !--author-handle-->@brendapraggastis<!--end-author-handle-- (Brenda Praggastis) Repository: https://github.com/pnnl/HyperNetX Branch with paper.md (empty if default branch): paper Version: v2.2.0p Editor: !--editor-->@danielskatz<!--end-editor-- Reviewers: @szhorvat, @IvanIsCoding, @drj11 Archive: 10.5281/zenodo.10795225
Status
Status badge code:
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
Reviewer instructions & questions
@szhorvat & @IvanIsCoding, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:
The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @danielskatz know.
✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨
Checklists
📝 Checklist for @szhorvat
📝 Checklist for @drj11
📝 Checklist for @IvanIsCoding