search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
709 stars 38 forks source link

[REVIEW]: AstronomicAL: an interactive dashboard for visualisation, integration and classification of data with Active Learning #3635

Closed whedon closed 3 years ago

whedon commented 3 years ago

Submitting author: @grant-m-s (Grant Stevens) Repository: https://github.com/grant-m-s/astronomicAL Version: v1.0 Editor: @crvernon Reviewer: @crhea93, @rmorgan10 Archive: 10.5281/zenodo.5396671

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/2dc5147d2c434806e2c8b8126c132104"><img src="https://joss.theoj.org/papers/2dc5147d2c434806e2c8b8126c132104/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/2dc5147d2c434806e2c8b8126c132104/status.svg)](https://joss.theoj.org/papers/2dc5147d2c434806e2c8b8126c132104)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@crhea93 & @rmorgan10, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @crvernon know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @crhea93

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @rmorgan10

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @crhea93, @rmorgan10 it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
whedon commented 3 years ago

PDF failed to compile for issue #3635 with the following error: 

 Can't find any papers to compile :-(
whedon commented 3 years ago 
Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.12 s (381.1 files/s, 99228.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          22           2341           1023           6664
reStructuredText                16            528            650            418
JSON                             3              0              0            290
Markdown                         1             20              0             31
Jupyter Notebook                 1              0            162             28
DOS Batch                        1              8              1             26
YAML                             2              3              5             18
make                             1              4              7              9
-------------------------------------------------------------------------------
SUM:                            47           2904           1848           7484
-------------------------------------------------------------------------------

Statistical information for the repository '42ba3ed564252039c49c17fb' was
gathered on 2021/08/20.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Grant Stevens                  226         16604           6576          100.00

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Grant Stevens             10028           60.4          3.4                2.76
crvernon commented 3 years ago

@whedon generate pdf from branch joss-paper-branch

whedon commented 3 years ago 
Attempting PDF compilation from custom branch joss-paper-branch. Reticulating splines etc...
whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

crvernon commented 3 years ago

@crhea93 @rmorgan10 @grant-m-s :wave: the review takes place in this issue. Thanks!

crvernon commented 3 years ago

Also, please don't forget to add a link to this review issue in any issues or pull requests you may generate in the https://github.com/grant-m-s/astronomicAL repository. This will help everyone have a single point of reference.

grant-m-s commented 3 years ago

Thanks @crvernon for setting this up and thanks @crhea93 and @rmorgan10 for agreeing to review AstronomicAL. If either of you have any questions I'll be happy to answer them.

rmorgan10 commented 3 years ago

Hi All!

@grant-m-s I would like to first congratulate you on an awesome software package! astronomicAL is really cool and a great addition to the field as a whole. I've identified 4 small things that will help me to check off all the remaining boxes above:

  1. Changes to the paper to clarify the state of the field and the statement of need in grant-m-s/AstronomicAL#9
  2. Updates to the installation instructions in grant-m-s/AstronomicAL#10
  3. Clarity on contributing and bug reporting in grant-m-s/AstronomicAL#11
  4. Verifying the functionalities in the testing suite in grant-m-s/AstronomicAL#12

On all these issues, let's iterate on them in your repo, and when they have been resolved you can close them.

Overall the tutorials are very thorough, the code is sufficiently documented, and I found astronomicAL easy to use. Great work!

crvernon commented 3 years ago

:mega: Mid-week rally! Just checking in to see how things are going.

🙌 Thanks for getting the ball rolling @rmorgan10 !

:open_hands: @crhea93 I am happy to answer any question you may have as well!

Great work on this review so far!

crhea93 commented 3 years ago

@grant-m-s WHAT A PACKAGE! This is extremely impressive work! The documentation is absolutely spot on -- I was able to answer any questions I had about the software and how best to use it by scrolling through the readthedocs pages.

I've added an additional installation issue to the existing GitHub issue created by @rmorgan10 grant-m-s/AstronomicAL#10

Since I am unable to install the software at the moment, I cannot complete my assessment of its functionality. Once we get that squared away, I'll be able to finish up my review :)

crvernon commented 3 years ago

@crhea93 I believe you accidentally linked to issue number 11 in the JOSS repository registry instead of one of the issues that is currently in the author's repository here: https://github.com/grant-m-s/AstronomicAL/issues

Would you mind editing your comment to reference the issue in the author's repo that you are referring to? Thanks!

crhea93 commented 3 years ago

My apologies! It should be all set now :D

grant-m-s commented 3 years ago

@whedon generate pdf from branch joss-paper-branch

whedon commented 3 years ago 
Attempting PDF compilation from custom branch joss-paper-branch. Reticulating splines etc...
whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

crvernon commented 3 years ago

:mega: Hello all! It looks like @crhea93 and @rmorgan10 have made great progress on the review! Also, great work @grant-m-s in addressing their issues as they arise.

Just a reminder for the reviewers: don't forget to check off the reviewer checklist as you feel your comments have been sufficiently addressed. Also, once you believe your review to be completed, shoot me a comment here.

👏 Great progress everyone and keep up the good work!

crhea93 commented 3 years ago

@crvernon My review is complete. I wholeheartedly suggest JOSS publishes this work!

crvernon commented 3 years ago

:clap: Thanks @crhea93 for your volunteering your time and providing a thoughtful review!

rmorgan10 commented 3 years ago

@crvernon Mine is all done too. Congrats @grant-m-s on a fantastic package!

crvernon commented 3 years ago

:clap: Thanks so much @rmorgan10 ! Your time and effort in reviewing this submission is greatly appreciated!

grant-m-s commented 3 years ago

Thank you @rmorgan10 and @crhea93 . I really appreciate your feedback and making my first review such a pleasant experience. Thanks again @crvernon managing everything.

grant-m-s commented 3 years ago

@crvernon what’s the next step in the process?

crvernon commented 3 years ago

@grant-m-s - I'll take a look at the paper document later today and build a list of any edits I think need to be made (if there are any).

Once I deem everything is good to go, I'll get you to generate a release and an accompanying DOI as linked with a service like Zenodo that corresponds to the final review version.

Lastly, once all has been accounted for, I'll make my recommendation to the editor-in-chief and they will conduct a final pass and move forward with full acceptance and publication if they are in agreement with my recommendation.

crvernon commented 3 years ago

@whedon generate pdf from branch joss-paper-branch

whedon commented 3 years ago 
Attempting PDF compilation from custom branch joss-paper-branch. Reticulating splines etc...
whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

crvernon commented 3 years ago

@whedon check references

crvernon commented 3 years ago

@grant-m-s your joss-paper-branch is 9 commits behind and 5 commits ahead of the master branch. Some of your changes per review comments were only made to the master and the paper directory resides on the joss-paper-branch. Are you planning on merging joss-paper-branch into master?

crvernon commented 3 years ago

@grant-m-s - In your Citing the Software documentation, you mention that:

astronomicAL has also been published as an arXiv_ preprint.

Could you please give me a link to that preprint?

grant-m-s commented 3 years ago

@crvernon That was added at the beginning of the docs being created as we assumed we would upload but we never actually did. Would you rather that I remove that?

grant-m-s commented 3 years ago

and I will merge the paper into master now.

crvernon commented 3 years ago

Yes, if the arXiv submission was never made it would be good to remove this to support the originality of this work. Thank you!

grant-m-s commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

grant-m-s commented 3 years ago

@crvernon all merged and updated the citation page.

crvernon commented 3 years ago

@whedon check references

whedon commented 3 years ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5281/zenodo.4581995 is OK
- 10.5281/zenodo.3987379 is OK
- 10.5281/zenodo.4573728 is OK
- 10.1051/0004-6361/201730763 is OK
- 10.1086/176166 is OK
- 10.3847/1538-4365/ab929e is OK

MISSING DOIs

- 10.3115/1613715.1613855 may be a valid DOI for title: An analysis of active learning strategies for sequence labeling tasks
- 10.1093/mnras/stz2816 may be a valid DOI for title: Galaxy Zoo: probabilistic morphology through Bayesian CNNs and active learning
- 10.1038/s41587-020-0521-4 may be a valid DOI for title: Active machine learning helps drug hunters tackle biology.

INVALID DOIs

- None
crvernon commented 3 years ago

@grant-m-s - Your paper is looking great! Here are several things I would like you to address from my review of your paper before we move on:

grant-m-s commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

grant-m-s commented 3 years ago

@crvernon so I've added all doi's I can find, its mainly the software packages that dont have a zenodo that I'm missing.

I've referenced all the figures in the text now.

Figure 1 is still ahead in the text as I believe it fits much better there rather than half way through the section. Let me know if that's against any style requirements.

Authors and Affiliations are all good.

crvernon commented 3 years ago

@grant-m-s - I think the figure referencing looks good.

crvernon commented 3 years ago

@whedon check references

whedon commented 3 years ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5281/zenodo.4581995 is OK
- 10.5281/zenodo.3987379 is OK
- 10.5281/zenodo.4573728 is OK
- 10.2200/S00429ED1V01Y201207AIM018 is OK
- 10.1051/0004-6361/201730763 is OK
- 10.1086/176166 is OK
- 10.3847/1538-4365/ab929e is OK
- 10.5281/zenodo.4724125 is OK
- 10.3115/1613715.1613855 is OK
- 10.1093/mnras/stz2816 is OK
- 10.1038/s41587-020-0521-4 is OK

MISSING DOIs

- None

INVALID DOIs

- 10.5555/1699510.1699549 is INVALID
- 10.5555/3305381.3305504 is INVALID
grant-m-s commented 3 years ago

hmm both seem to be ACM doi's that aren't matched anywhere else besides the ACM website.

should I remove them?

crvernon commented 3 years ago

Yes, please remove them. It looks like those repos only site the tag on GitHub and not a DOI. Please pull the two invalid DOI links and we will proceed.

grant-m-s commented 3 years ago

ok they've now been removed

crvernon commented 3 years ago

@whedon check references