[REVIEW]: fishRman: A Shiny R Dashboard improving Global Fishing Watch data availability

whedon commented 3 years ago

Submitting author: @Shyentist (Pasquale Buonomo) Repository: https://github.com/Shyentist/fish-r-man Version: v1.0.0 Editor: @KristinaRiemer Reviewer: @jules32, @HeatherWelch Archive: 10.5281/zenodo.5582567

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/9895e0e14fbd2be0622022dbbe15fe10"><img src="https://joss.theoj.org/papers/9895e0e14fbd2be0622022dbbe15fe10/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/9895e0e14fbd2be0622022dbbe15fe10/status.svg)](https://joss.theoj.org/papers/9895e0e14fbd2be0622022dbbe15fe10)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@jules32 & @HeatherWelch, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @KristinaRiemer know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @jules32

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@Shyentist) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @HeatherWelch

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@Shyentist) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @jules32, @HeatherWelch it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 3 years ago

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.05 s (709.7 files/s, 88755.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JSON                            19             27              0           1923
R                               11            265             63           1758
Markdown                         5             46              0            165
CSS                              1             85              0            147
TeX                              1             16              0            132
-------------------------------------------------------------------------------
SUM:                            37            439             63           4125
-------------------------------------------------------------------------------

Statistical information for the repository '1f71e8d768c9244d0ccab880' was
gathered on 2021/07/09.
No commited files with the specified extensions were found.

whedon commented 3 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.21105/joss.01686 is OK
- 10.32614/RJ-2018-009 is OK
- 10.5281/zenodo.4679424 is OK

MISSING DOIs

- 10.21105/joss.00848 may be a valid DOI for title: countrycode: An R package to convert country names and country codes

INVALID DOIs

- None

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

KristinaRiemer commented 3 years ago

Hi @Shyentist, thanks again for your JOSS submission! This is to notify you that I'm going to be on vacation July 14 - August 8. Hopefully you and the submission reviewers @jules32 and @HeatherWelch will be able to make progress on this submission's review in my absence. If anything urgent comes up during that time period, feel free to contact EiC @arfon. If you have any questions right now, please let me know.

KristinaRiemer commented 3 years ago

/ooo July 14 until August 8

whedon commented 3 years ago

:wave: @jules32, please update us on how your review is going (this is an automated reminder).

whedon commented 3 years ago

:wave: @HeatherWelch, please update us on how your review is going (this is an automated reminder).

jules32 commented 3 years ago

Hi! I'll be reviewing by mid-week next week.

jules32 commented 3 years ago

@whedon commands

whedon commented 3 years ago

Here are some things you can ask me to do:

# List Whedon's capabilities
@whedon commands

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

EDITORIAL TASKS

# Compile the paper
@whedon generate pdf

# Compile the paper from alternative branch
@whedon generate pdf from branch custom-branch-name

# Ask Whedon to check the references for missing DOIs
@whedon check references

# Ask Whedon to check repository statistics for the submitted software
@whedon check repository

jules32 commented 3 years ago

👋 @arfon @whedon

I'm logged in but unfortunately it looks like my review invitation has expired. I know Kristina is out, would you be able to help?

https://github.com/openjournals/joss-reviews/invitations

danielskatz commented 3 years ago

@whedon re-invite @jules32 as reviewer

whedon commented 3 years ago

OK, the reviewer has been re-invited.

@jules32 please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

HeatherWelch commented 3 years ago

@whedon it seems like my review invitation has also expired

[image: Screen Shot 2021-07-26 at 7.25.23 AM.png]

On Sun, Jul 25, 2021 at 5:03 PM whedon @.***> wrote:

OK, the reviewer has been re-invited.

@jules32 https://github.com/jules32 please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/3467#issuecomment-886279815, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFTRCYECOFHOZITEKJUISTDTZSQ3TANCNFSM5ADNDNKQ .

--

Heather Welch Project Scientist

UCSC and NOAA SWFSC Environmental Research Division 99 Pacific St. Suite 255A Monterey, CA 93940

phone (wk): 831-648-8516 http://conservationplanning.org/people/heather-welch/ http://conservationplanning.org/people/heather-welch/ https://github.com/HeatherWelch

whedon commented 3 years ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands

danielskatz commented 3 years ago

@whedon re-invite @HeatherWelch as reviewer

whedon commented 3 years ago

OK, the reviewer has been re-invited.

@heatherwelch please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

HeatherWelch commented 3 years ago

@Shyentist a couple questions/issues:

I am not able to get the visualize button to work, either by converting the currently queried csv or by uploading a .gpkg.
Would it be possible to clear the forms on the analysis page when starting a new analysis? E.g. in this situation I am not sure if I'm analyzing the current query or the uploaded .gpkg:

Shyentist commented 3 years ago

Hello @HeatherWelch .

What do you mean that the button does not work? Does it remain greyed out? Does it do nothing upon clicking? I am unable to replicate the issue.
As of now, no, but that would be a nice addition indeed. EDIT: The analyses are always performed on the latest data loaded onto the Analysis tab.

jules32 commented 3 years ago

Hello @Shyentist,

I’ve completed my first review, thank you for submitting. Below are some comments that I think would benefit your project; they are largely about documentation in various forms.

The Shiny App launched very quickly and was very responsive, and this seems like it will be helpful for researchers wanting to query and analyze Global Fishing Watch data. I think it would benefit from more documentation in the App itself; maybe a landing page that introduces Global Fishing watch with links, describes how people might query and use their data, and describes the data with clear metadata that the user will encounter. Here it would also be appropriate to link the fisheRman Handbook PDF that describes the App’s usage.

Then, on the Query and Analysis tabs, small examples of how to use the App would be very helpful. This could be pre-loaded so that the user can see the example and then clear it to begin their own query. (My first attempts resulted in either “Your query would return more than 1000000 rows” or “Showing 0 to 0 of 0 entries” as I was trying to familiarize myself with the functionality.)

The code itself (global.R, server.R, ui.R) could benefit from more documentation so that it is more clear how all the code works together and so that it could be easier for contributors to navigate and provide any suggestions.

In the paper, I think the statement of need and the state of the field could be more detailed; are there citations and descriptions for who is using the data, how they are using it, what issues they are encountering, and how fisheRman helps? Who is the desired user, what do they have to know, how will this integrate into their workflow?

The fisheRman handbook is very useful; I think it would benefit the user if it were linked from the repo’s README, from the App’s landing page, and from the paper.

Shyentist commented 3 years ago

Hello @jules32 , thank you for your feedback and advice.

Embedding the handbook in the app was planned a couple of days after my submission, when I saw another app doing the same with its own instructions. I was waiting for the review process to end before doing that. Definitely to-do.
Also, I am aware of the scarcity of comments in the code. Improving that is also definitely in the to-do list.
Handbook linked from the repo's README: Definitely to-do.
About the in-app examples and the information on the metadata, I would prefer adding them to the handbook, rather than having them pop-up or be pre-loaded only to be cleared by the user. That is, because I am fairly sure those would be annoying for someone who already knows how the app works, while they would be ignored by anyone who hasn't bothered reading any documentation in the first place. I will try to figure out something that impacts less on the UX. Any suggestion is appreciated.
About the paper, I will try my best to answer those questions.

How do we proceed now? Do I make the adjustment and the review goes on from there when I do, or is there anything else with the bot?

HeatherWelch commented 3 years ago

Hi @Shyentist,

I have completed my review. Nice work on making GFW data more accessible and easy to manipulate. Below are a few questions and suggestions for improvement.

It would be good to provide more background on the GFW dataset for people who are not familiar with GFW products already. E.g. what is AIS, what are the dimensions of the data (size, extent, date range), brief introduction of GFW's neural net used to identify gear types and fishing vs non-fishing behaviors (and link to Kroodsma et al 2018), what is the difference between fishing and vessel hours, published examples of the data being used to address real-world problems.
I found the functionality described in the PDF very straightforward and useful.
Fishing effort is tricky as a catch all term - this dataset also includes non-fishing behaviors (vessel hours). I wonder if there's a broader term that would encompass both fishing and non-fishing behaviors, perhaps just 'AIS data'.
Would greatly benefit from an example of real-world problem solving. There were a lot of big news stories last year about Chinese vessels fishing the border of the Ecuadorian EEZ near the Galapagos. It would be neat to use this as a prompt to guide an analysis example.
Is 'References' under paper.md meant to be blank?

Query tab. -Returned table should have column names that match the filter terms, e.g. "hours" in the table should match "vessel_hours" in the filter terms, "cell_II_lat" should match "Latitude", etc. -For the Flag filter, please link to an ISO look up table and note that these are Alpha 3 Codes. This will be helpful for being who don't know the code of their country of interest.
-For fishing effort at 10th degree there is an option to filter by MMSI; for fishing effort at 100th degree there is an option to filter by MMSI present. It is not clear to me why or how these are different. In the handbook, it would be helpful to have more information on the difference between MMSI and MMSI present filters, and why different options are provided depending on resolution.

Analysis tab.
-It would be helpful if the date range was included somewhere in the map. When writing out a png, it's helpful to maintain information on what time-series e.g. total fishing activity is calculated.
-I am still having trouble with the spatial visualization. It occurs when I convert the current data. Sometimes it doesn't happen, sometimes it does. I just had the error occur with this order of operations:

run the pre-filled filter on the query tab.
convert the current data on the analysis tab.
summarize by year on the descriptive tab.
visualize on the spatial tab (works).
summarize by year and cell_II_lon on the descriptive tab.
visualize on the spatial tab (nothing happens).
I have a feeling the data frame is being modified during the summary, and when you collapse by longitude there are no longer meaningful lat/lon coordinates to use for mapping.

arfon commented 3 years ago

How do we proceed now? Do I make the adjustment and the review goes on from there when I do, or is there anything else with the bot?

@Shyentist – please make your updates and let us know here when you are ready for the reviewers to take another look. You can update your paper preview at any time with @whedon generate pdf

Shyentist commented 3 years ago

@arfon Great, thank you!

@HeatherWelch Hi again, thank you for your review and the useful feedback. I am assuming the first list concerns the Handbook, not the paper, apart from the last point.

While the description of the data for each column would be useful and will be added, I feel like a thorough description on how the dataset was created is beyond the scope of the Handbook, the paper, and the app itself. I will, however, surely link to the work of Kroodsma for more details.
Thank you, I try my best to make it user-friendly and understandable for non-programmers.
Correct, I will amend accordingly.
Definitely. I will work on it.
Yes, the bot generates the pdf with paper.md up to the "References" section, then fills in the entries with the content of paper.bib.

Query tab

I will explain the Filter name <--> Column name matches in the Handbook, in the section where each column will be explained, but I would prefer keeping them as they are, to avoid scaring users away with filters such as "cell_ll_lon" that looks much less friendly than "Longitude".
I love this idea, so simple and yet so needed.
Yes, definitely.

Analysis tab

Definitely.
That is actually how the app is supposed to behave. The Visualize button and results are completely unrelated to the Summarize button and table. Visualize creates a latitude-longitude plot, so a map, with the spatial data provided, and is not meant to be a "plotter" for what is summarized. Summarize, on the other hand, is an aggregator meant to create a new table that can be downloaded as a csv for whatever need the user might have. Basically, Visualize does not operate on the summarized dataframe, which does not always retain lat-lon information and can reach up to 7 different dimensions, but on the original spatial data.

KristinaRiemer commented 3 years ago

Thanks for providing reviews during my absence, @jules32 and @HeatherWelch! Just to confirm what @arfon said, @Shyentist you should make changes as suggested by the reviewers and feel free to update us when complete or discuss these more here!

KristinaRiemer commented 3 years ago

@Shyentist just checking in to see how your updates are going...let me know if you need anything!

Shyentist commented 3 years ago

@KristinaRiemer hi, I have made some improvements for what was pointed out during the first review, and I am working on the actual paper as of now. I will surely let you know in case I need anything, thank you!

Shyentist commented 3 years ago

@KristinaRiemer hi, I have a question about the paper. One of the points in jules' review was

The fisheRman handbook is very useful; I think it would benefit the user if it were linked from the repo’s README, from the App’s landing page, and from the paper.

I was wondering, if the url for the Handbook changes after the paper, wouldn't that be an issue? What are the rules for updating the paper, if that is even possible?

KristinaRiemer commented 3 years ago

Are you expecting the handbook URL to change? I think probably as long as that URL is maintained in the GitHub repository README, users will be able to find it.

Shyentist commented 3 years ago

I think so. If the documentation expands beyond the Handbook, I would like to arrange it in folders as the project goes on. I think the best course of action would be to link to the repository itself, with the README always bearing the updated URL for the Handbook (and whatever else might be needed).

Shyentist commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

Shyentist commented 3 years ago

@KristinaRiemer I think the repository is ready for a second review. Is there something for me to do?

KristinaRiemer commented 3 years ago

Thanks @Shyentist! The next step would be for @jules32 and @HeatherWelch to review the changes you've made, update the review checklists accordingly, and identify if there is anything remaining that could be addressed.

jules32 commented 2 years ago

Hi @shyentist and @KristinaRiemer, I'll be able to do a second review by next Monday; sorry for the delay here but I haven't forgotten!

KristinaRiemer commented 2 years ago

Hi @jules32, thanks so much for checking in and providing a deadline!

jules32 commented 2 years ago

Hi @Shyentist , I’ve re-reviewed and think that you have met most of the checklist required for the JOSS review at this point. You've addressed my previous comments and I think that the Handbook has improved a lot; the Example on p. 12 was especially helpful for understanding how the App could be used. The only thing that I didn’t see in the Software Paper was the “State of the Field”; perhaps https://globalfishingwatch.org/data/working-with-our-downloadable-public-data-in-r/ would be a potential citation for this section.

In the long term (beyond the scope of this JOSS review), I do think that more documentation focused on your primary audience would increase the number of users who use the App, and improve their experience with it. For example, I think the App itself would benefit greatly from some sort of text/banner on the home page that says something like “This is the fishRman app to help you explore and analyze Global Fishing Watch Data. Learn more about Global Fishing Watch and the data available for querying with these links”. I say this because I expect that the App will be the “first contact” for many fisheries researchers, who learn about it from their colleagues who have shared the https://shyentist.shinyapps.io/fish-r-man/ link. Anyone who receives this link will have not read the manual, so having a title and information about what is up top will be very helpful. And one last thought – I have recently learned about https://documentation.divio.com/ and found it to be really helpful for thinking about writing software documentation and am sharing it here in case it’s useful for you too.

Cheers, Julie

HeatherWelch commented 2 years ago

Hi @Shyentist, I have also completed my 2nd review. Nice job on revisions, you have addressed all of my comments. I particularly like the in-depth Galapagos example in the handbook (really nicely done), and that the handbook is now directly embedded in the app. I have no further edits. Best, Heather

Shyentist commented 2 years ago

Thank you @jules32 and @HeatherWelch for the 2nd review. I'll add an explanatory banner redirecting to the proper documentation and amend the state of the field as per suggestion. The link provided is, in fact, the one I often refer to when met with an issue with GFW data. About the documentation, thank you for Divio, I'll look into that, although I started reading {officedown}, to solve the same issue. I'll see which fares better.

Cheers 😄

Shyentist commented 2 years ago

@whedon generate pdf

whedon commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

KristinaRiemer commented 2 years ago

Thanks for the followup reviews @jules32 and @HeatherWelch. @jules32, that is a great documentation reference you've provided! It looks like @HeatherWelch is satisfied with the current state of the software, and @Shyentist once you've made those last changes, please let us know so that @jules32 can provide more followup comments or check off on this if the changes are sufficient.

Shyentist commented 2 years ago

Hi, yes, I think the paper is ready for examination @KristinaRiemer @jules32

I added the introductory message with hyperlinks and the state of the field to the best of my knowledge.

jules32 commented 2 years ago

Looks good to me @Shyentist , great job 🥳

Cheers, Julie

KristinaRiemer commented 2 years ago

Excellent! Thanks so much to @jules32 and @HeatherWelch for your thoughtful reviews!

@Shyentist, for next steps, I'll be proofreading and checking references of the paper (which you should feel free to do so also). After that looks good, we'll have you archive the code.

KristinaRiemer commented 2 years ago

@whedon generate pdf

whedon commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

KristinaRiemer commented 2 years ago

@whedon check references

whedon commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.21105/joss.01686 is OK
- 10.32614/RJ-2018-009 is OK
- 10.5281/zenodo.4679424 is OK

MISSING DOIs

- 10.21105/joss.00848 may be a valid DOI for title: countrycode: An R package to convert country names and country codes
- 10.1002/ecs2.2753 may be a valid DOI for title: Enhancing collaboration between ecologists and computer scientists: Lessons learned and recommendations forward.
- 10.1002/ecs2.2567 may be a valid DOI for title: Evaluating the popularity of R in ecology.

INVALID DOIs

- None

openjournals / joss-reviews