search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
722 stars 38 forks source link

[REVIEW]: GeoClimate: a Geospatial processing toolbox for environmental and climate studies #3541

Closed whedon closed 3 years ago

whedon commented 3 years ago

Submitting author: @ebocher (Erwan Bocher) Repository: https://github.com/orbisgis/geoclimate/ Version: 0.0.1 Editor: @elbeejay Reviewer: @abhishekvp, @omshinde, @arbennett Archive: 10.5281/zenodo.5534680

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/2b216ea704e5ee9d61a8208463c17ce1"><img src="https://joss.theoj.org/papers/2b216ea704e5ee9d61a8208463c17ce1/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/2b216ea704e5ee9d61a8208463c17ce1/status.svg)](https://joss.theoj.org/papers/2b216ea704e5ee9d61a8208463c17ce1)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@abhishekvp & @omshinde & @arbennett, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @elbeejay know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @abhishekvp

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @omshinde

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @arbennett

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @abhishekvp, @omshinde, @arbennett it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
whedon commented 3 years ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1023/A:1002463829265 is OK
- 10.1175/BAMS-D-11-00019.1 is OK
- 10.1016/j.uclim.2018.01.008 is OK
- 10.1016/j.uclim.2019.100536 is OK
- 10.1016/j.buildenv.2017.08.009 is OK
- 10.3389/fenvs.2021.637455 is OK
- 10.1016/j.envsoft.2017.09.020 is OK
- 10.1016/j.uclim.2015.04.001 is OK
- 10.1002/met.29 is OK
- 10.1007/s00704-015-1405-2 is OK

MISSING DOIs

- None

INVALID DOIs

- None
whedon commented 3 years ago

Wordcount for paper.md is 2368

whedon commented 3 years ago 
Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=1.28 s (74.8 files/s, 118855.5 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
XML                              5             14             16          78777
SVG                              1              1              1          44890
Groovy                          37           1848           3232          15668
JSON                            32              5              0           4734
Scheme                           3              1              0            799
Maven                            5             29              7            761
SQL                              6            196            297            702
Markdown                         4             84              0            219
TeX                              1             11              0            147
YAML                             2              4              4             38
-------------------------------------------------------------------------------
SUM:                            96           2193           3557         146735
-------------------------------------------------------------------------------

Statistical information for the repository '2f3167a49ed43e063d1d57ef' was
gathered on 2021/07/26.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Bocher                           2             4              4            0.12
Elisabeth Le Saux                2           626            409           15.44
Palominos Sylvain                4             8             50            0.87
ebocher                         25          1389           1029           36.07
gpetit                          22          1652            343           29.76
jeremy                           1            22             22            0.66
jeremy-b                        50           604            441           15.59
nahtanojoal                      4            94              6            1.49

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Bocher                      368         9200.0          7.1                3.53
gpetit                      623           37.7         24.7               44.78
jeremy-b                    202           33.4         17.8                2.48
nahtanojoal                   2            2.1         26.5                0.00
whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

elbeejay commented 3 years ago

@abhishekvp, @omshinde, and @arbennett, thanks for agreeing to review this submission to JOSS. We are currently asking reviewers to try and complete their reviews in 6 weeks.

The JOSS review process is entirely open and transparent, and takes place on GitHub. Review comments can be made as issues in the GeoClimate repository, please link this review issue when doing so (paste https://github.com/openjournals/joss-reviews/issues/3541 into the issue).

For reference, here are links to the JOSS documentation that may be helpful as you conduct your reviews:

Please feel free to ping me (@elbeejay) if you have any questions/concerns. Thanks again for agreeing to review for JOSS.

whedon commented 3 years ago

:wave: @abhishekvp, please update us on how your review is going (this is an automated reminder).

whedon commented 3 years ago

:wave: @arbennett, please update us on how your review is going (this is an automated reminder).

whedon commented 3 years ago

:wave: @omshinde, please update us on how your review is going (this is an automated reminder).

elbeejay commented 3 years ago

:wave: Hello @abhishekvp @arbennett @omshinde we're at the halfway point for this review so I wanted to just check in with you all. Please feel free to reach out if you have any questions about the JOSS review process.

omshinde commented 3 years ago

Hi @elbeejay Thanks for the reminder. I have started preliminary review locally in my system but I am afraid that I am not able to update my checklist. Maybe because I am not assigned or added as collaborator. Please check it once so that I can update the checklist too (I hope I am not missing anything). Thanks!

elbeejay commented 3 years ago

@whedon re-invite @omshinde as reviewer

whedon commented 3 years ago

The reviewer already has a pending invite.

@omshinde please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

elbeejay commented 3 years ago

Hopefully the new invitation to review will give you access to the checklist @omshinde :crossed_fingers:.

omshinde commented 3 years ago

Thanks for the follow-up and really sorry for missing out on accepting the invitation. I tried accepting it now but it says that the invitation was revoked or I am not logged in with correct account (which I double checked and is not the case).

Screenshot from 2021-08-17 01-00-30

elbeejay commented 3 years ago

Thanks for following up @omshinde, if you cannot access the checklist please continue to perform your review anyway. You can copy the checklist items and make a comment in this issue if you'd like to have a way to keep track. Otherwise just open issues if/as necessary in the project repository, and let me know here when you've completed your review. Thanks!

omshinde commented 3 years ago

Review checklist for @omshinde

Conflict of interest

elbeejay commented 3 years ago

:wave: @abhishekvp @arbennett @omshinde just checking in here now that we are 5 weeks into the review period. Please let me know if you will be unable to finish your review by September 6th.

As always, ping me with any questions about the JOSS process.

omshinde commented 3 years ago

Hi @elbeejay Thanks for the reminder. I am on track with the review timeline and would submit it before Sep 6th.

arbennett commented 3 years ago

Hey @elbeejay I am also on track with the review and should be able to finish up in the next couple days. Thanks for checking in!

omshinde commented 3 years ago

@elbeejay @ebocher

Review 1: Thanks to the authors for their contributions. Please find my queries and suggestions below regarding the submission - "GeoClimate: a geospatial processing toolbox for environmental and climate studies".

Working Environment: OS - Ubuntu 20.04.2 LTS

Functionality

Query: The download link as mentioned on https://github.com/orbisgis/geoclimate/wiki/Download#geoclimate-for-command-line-interface for the Geoclimate.jar redirects to file not found for me. Could you please recheck and confirm? I hope I am not missing anything here. Thanks!

I will continue my review for functionality and documentation after receiving response in this regard.

Software Paper

References

  1. Minor Suggestion - The authors could consider citing the UMEP manual instead of putting the link as footnote in the Software paper. 1.1 (Please refer: https://umep-docs.readthedocs.io/en/latest/Introduction.html#umep-how-to-cite)

    Lindberg F, Grimmond CSB, A Gabey, L Jarvi, CW Kent, N Krave, T Sun, N Wallenberg, HC Ward (2019) Urban Multi-scale Environmental Predictor (UMEP) Manual. https://umep-docs.readthedocs.io/ University of Reading UK, University of Gothenburg Sweden, SIMS China

ebocher commented 3 years ago

@omshinde Thank you for pointing out this error. The page has been updated and the link is now pointing to the releases page.

abhishekvp commented 3 years ago

Hi @elbeejay Thanks for the reminder. I too, am on track with the review timeline and would turn in my review in a day. Thanks!

elbeejay commented 3 years ago

:wave: @abhishekvp @arbennett @omshinde wanted to ask how the reviews are coming along and remind you all to report any review comments or suggestions as GitHub issues in the GeoClimate repository. When doing so just remember to tag the JOSS review issue (this thread) so that it is referenced here.

As we are beyond the 6-week target deadline for reviews, please provide new estimates for when you expect to complete your first evaluations of this submission so that @ebocher and co-authors can be aware and prepared.

omshinde commented 3 years ago

@elbeejay Hi! Sure, will keep in mind to tag this issue. Now that I can access the installation file, I shall be able to submit my final reviews before this weekend. Thanks for the reminder and apologies for the delay.

omshinde commented 3 years ago

Hi @elbeejay! I am done with my reviews based on the above checklist. Please find my comments below. After the minor suggestions are addressed, I am good to approve the submission for publication with JOSS based on the review checklist. Thanks!

Working Environment

OS: Ubuntu 20.04.2 LTS Java version: openjdk version "1.8.0_292"

Submission

The authors presented a geospatial processing toolbox for environmental and climate studies named as GeoClimate. GeoClimate enables computation of climate and environmental paramaters such as Sky View Factor, urban classifications such as Local Climate Zones based on vector based inputs. The submission shows the examples using OpenStreetMaps and French BD Topo 2.2 version databases.

Functionality

The installation instructions works perfectly and are easy to follow. The tutorials cover the details and contains step-by-step instructions to execute the algorithms. I could run the examples for the default city of Pont-de-Veyle. In order to check the reproducibility and efficacy of GeoClimate, the instructions were followed for a city in India - Mumbai. Additionally, it was also tested using the bounding box as input for the OSM workflow. Please find the outputs (screenshots of visualization of generated GeoJSONs using QGIS) below for these cases to validate if they are correct -

Screenshot from 2021-09-11 16-44-30

Screenshot from 2021-09-11 17-34-49

But, while executing the Urban Typology by Random Forest (UTRF) classification, I faced some issues. The output log follows as here. (Note: I did trying solving this issue by following the hack mentioned in the FAQs)

The Documentation is more than sufficient to get started working with GeoClimate. Though the API methods documentation is missing but I feel that could be added in future contributions. The Software paper is very well drafted as well.

Some minor suggestions:

  1. In the Bounding box tutorial, the description for Bounding box method with BDTOPO v2.2 could be removed if it is not ready yet. There is only a heading with no further steps to follow.

  2. The authors could consider citing the UMEP manual instead of putting the link as footnote in the Software paper. 1.1 (Please refer: https://umep-docs.readthedocs.io/en/latest/Introduction.html#umep-how-to-cite)

Lindberg F, Grimmond CSB, A Gabey, L Jarvi, CW Kent, N Krave, T Sun, N Wallenberg, HC Ward (2019) Urban Multi-scale Environmental Predictor (UMEP) Manual. https://umep-docs.readthedocs.io/ University of Reading UK, University of Gothenburg Sweden, SIMS China

My remarks: GeoClimate provides a great utility in computing the environmental and climate related parameters based on several indicators. Thanks to the authors for their efforts and for providing a neat software package to the scientific community.

Kind regards, Rajat

abhishekvp commented 3 years ago

Hey @elbeejay! Thanks for the reminder. I have already checked some items off the the review checklist. I am working towards setting up the processing toolbox locally and testing it. I shall be able to complete the review by Tuesday, September 14, 2021 at the latest if not earlier. I apologize for the delay in turning in my review.

Thanks.

arbennett commented 3 years ago

Hi @elbeejay - thanks for the reminder. My completed review is below.

@ebocher - thanks for all your (and your team's) efforts in developing GeoClimate. While a bit outside of my own area of expertise I was able to understand the goals and & reasons for developing GeoClimate quite easily. I was also able to run through some of the examples that you've provided in your documentation. Overall I think the paper was well written and the software functions well. I have only a couple of minor things that I would like to see changed. They are below:

ebocher commented 3 years ago

@abhishekvp 

* Some of the links in the documentation were not working:  JOSS review: broken links in docs orbisgis/geoclimate#623

The documentation you mention is ouf of dated. Can you tell me how you found it, from a link in the wiki doc? So I will be able to fix it.

ebocher commented 3 years ago

But, while executing the Urban Typology by Random Forest (UTRF) classification, I faced some issues. The output log follows as here. (Note: I did trying solving this issue by following the hack mentioned in the FAQs)

@omshinde Thanks for the stack trace. It seems a pb with your network. Could you check if you have access to https://github.com/orbisgis/geoclimate/raw/master/models/URBAN_TYPOLOGY_BDTOPO_V2_RF_2_1.model GeoClimate downloads this file to install it on your machine and run the machine learning algorithm.

ebocher commented 3 years ago

@omshinde

Though the API methods documentation is missing but I feel that could be added in future contributions. GeoClimate is a set of Groovy scripts so you are right there is no API but all scripts are fully documented. Here an example to apply the machine learning algorithm : https://github.com/orbisgis/geoclimate/blob/5bd90c2eeb36e626e5caf1b64bdae83c668e446a/geoindicators/src/main/groovy/org/orbisgis/geoclimate/geoindicators/TypologyClassification.groovy#L565

ebocher commented 3 years ago 
1. In the [Bounding box tutorial](https://github.com/orbisgis/geoclimate/wiki/Bounding-box-case-%5BL%5D), the description for [Bounding box method with BDTOPO v2.2](https://github.com/orbisgis/geoclimate/wiki/Bounding-box-case-%5BL%5D#bounding-box-method-with-bdtopo-v22) could be removed if it is not ready yet. There is only a heading with no further steps to follow.

The tutorial has been updated to integrate BDTOPO v2.2 BBOX case. see https://github.com/orbisgis/geoclimate/wiki/Bounding-box-case-%5BL%5D

ebocher commented 3 years ago

@elbeejay Comments from @arbennett and @omshinde have been fixed. We are now waiting for @abhishekvp

elbeejay commented 3 years ago

@elbeejay Comments from @arbennett and @omshinde have been fixed. We are now waiting for @abhishekvp

Great! Just to let you know how we will proceed, after all comments have been addressed I will reach out to the reviewers one last time so that they have a chance to do a final check and give their approval that the submission is ready to be published.

omshinde commented 3 years ago

But, while executing the Urban Typology by Random Forest (UTRF) classification, I faced some issues. The output log follows as here. (Note: I did trying solving this issue by following the hack mentioned in the FAQs)

@omshinde Thanks for the stack trace. It seems a pb with your network. Could you check if you have access to https://github.com/orbisgis/geoclimate/raw/master/models/URBAN_TYPOLOGY_BDTOPO_V2_RF_2_1.model GeoClimate downloads this file to install it on your machine and run the machine learning algorithm.

@ebocher I think this is the issue as it seems to be a SSL connection error. Thank you for addressing this.

omshinde commented 3 years ago

@ebocher Thank you for addressing the comments. @elbeejay I am ready with my final review and waiting for your further instructions.

elbeejay commented 3 years ago

Hi @omshinde, thank you for the constructive review! Once the other reviewers have finished their reviews and their comments have been addressed, I will reach out to simply confirm that you believe this submission is ready for publication. This gives you the opportunity (if you are able) to have a brief look at any new changes that have been made since your review was completed, and is the final opportunity to make any further suggestions.

Thanks again for your review, I will ping you when we reach this next step in the process.

abhishekvp commented 3 years ago

@ebocher @elbeejay Please find my review and comments below:

Thanks to the authors for their contribution to the Open Source Community and for their interest in publishing their work in the Journal of Open Source Software.

Research Paper

The paper is very well organized. It discusses the GeoClimate processing toolbox that has been developed to compute complex morphological indicators and urban classifications. The need for such a tool in the urban studies research area has been justified. The authors also put forth a sound comparison of GeoClimate with UMEP and the LCZ Generator - the other prominent tools in this research area. The discussion on the coding implementation, configuration and the output is elaborate with the relevant details being covered. The authors list the relevant references that have been referred for this work. Overall the paper is excellently articulated and seems a great fit for the Journal of Open Source Software.

Code Repository and Documentation

I was able to locally test the GeoClimate library on my system.

Working Environment: OpenJDK Runtime Environment (build 11.0.11+9-Ubuntu-0ubuntu2.20.04)

The tutorial documentation on the repository's wiki page is comprehensive and serves the purpose. I was able to successfully test the GeoClimate library for - (1) OSM with Locality value as a parameter and (2) OSM with Bounding Box. The indicators and classifications were successfully generated as desired in both cases.

I have a couple of minor suggestions which I would like the authors to address, before we go ahead with approving the paper for publication:

  1. Although the content for the header - Bounding box method with BDTOPO v2.2 has been added, as rightly pointed out by @omshinde in his review, the configuration snippet in the description field still says Processing OSM data. I believe it would need to be modified to Processing BD Topo v2 data for ensuring consistency.
  2. I observed that the tutorial pages for - (1) BDTopo 2.2 and (2) Bounding box case for the Windows operating system, have not been created and simply point to the Wiki Home page of the repository. To ensure consistency in the documentation, the authors are encouraged to either (1) create these 2 pages - they may have contain a 'Work in Progress' message for now and the actual content can be populated when it is ready or (2) delete the pages from the navigation panel.

In conclusion, I would like to convey my sincere gratitude to the authors for their valuable contribution to the Open Source community. I am positive that GeoClimate would find great use in numerous applications for researchers working in the area of Urban Climate Studies.

ebocher commented 3 years ago

Dear @abhishekvp Thanks for the comments. All tutorial pages have been updated and there is no a documentation the Windows OS.

@elbeejay It seems we are now on the road. All comments, suggestions, corrections have been done in the paper or in the documentation.

abhishekvp commented 3 years ago

@ebocher Thank you for addressing the comments!

elbeejay commented 3 years ago

Hi all, thanks for reviewing this work, and thank you to @ebocher and co-authors for making revisions as necessary.

At this time I would like to confirm with @omshinde, @arbennett and @abhishekvp that the revised software and documentation in its current form is acceptable to you. At this time one outstanding review issue here has yet to be closed, so @arbennett please take a look at the revisions made since that was opened and close the issue or make additional suggestions as necessary.

Thanks again all for your contributions to this review process.

omshinde commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

omshinde commented 3 years ago

Hi @elbeejay, @ebocher Thank you to the authors for addressing the comments patiently and promptly. I am very happy with the submission and feel elated to accept the "GeoClimate: a Geospatial processing toolbox for environmental and climate studies" for publication in JOSS. Congratulations to the authors for their efforts in making a great contribution to the scientific community.

@elbeejay Thanks for the opportunity to review this submission. Kind regards, Rajat

arbennett commented 3 years ago

Hi @elbeejay & @ebocher - I am also happy to say that I am ready to accept this submission for publication in JOSS! Thanks again to @ebocher & the rest of the team for the quick turnaround, great work!

elbeejay commented 3 years ago

Thanks again to our reviewers. @ebocher and co-authors I've opened a minor PR recommending some slight edits to the grammar of the paper, please review and decide if you would like to incorporate those.

Once that decision is made, there are just a few items that need to be taken care of:

Thanks, please just reply with the above items once they are complete and then I will be able to send this on to the EiC for publication.

abhishekvp commented 3 years ago

Hi all, thanks for reviewing this work, and thank you to @ebocher and co-authors for making revisions as necessary.

At this time I would like to confirm with @omshinde, @arbennett and @abhishekvp that the revised software and documentation in its current form is acceptable to you. At this time one outstanding review issue here has yet to be closed, so @arbennett please take a look at the revisions made since that was opened and close the issue or make additional suggestions as necessary.

Thanks again all for your contributions to this review process.

Hi All!

I too am happy to state that the revised software and documentation in its current form is acceptable to me.

@elbeejay I am grateful to you for the opportunity to review this work.

ebocher commented 3 years ago

Hi @elbeejay

Make a tagged release of the software and report the version tag for the archived code here

This is done here https://github.com/orbisgis/geoclimate/releases/tag/0.0.1 We are working to compile also the doc with the release.

Archive the reviewed code on Zenodo or a similar service (e.g., figshare, an institutional repository) Check that the archived code (e.g., Zenodo) has the correct metadata. The title should match the paper title, the author list should be correct, and you can include authors' ORCID if you wish

We plan to use https://www.softwareheritage.org but I don't know if it is able to generate a DOI.

Provide the DOI of the archived code here

elbeejay commented 3 years ago

Make a tagged release of the software and report the version tag for the archived code here

This is done here https://github.com/orbisgis/geoclimate/releases/tag/0.0.1 We are working to compile also the doc with the release.

Can you make a tagged release of the current codebase? 27 commits have been made since 0.0.1 and I think at least 1 modified the source code.

Archive the reviewed code on Zenodo or a similar service (e.g., figshare, an institutional repository) Check that the archived code (e.g., Zenodo) has the correct metadata. The title should match the paper title, the author list should be correct, and you can include authors' ORCID if you wish

We plan to use https://www.softwareheritage.org but I don't know if it is able to generate a DOI.

@openjournals/joss-eics can https://www.softwareheritage.org be used for this purpose?

arfon commented 3 years ago

@openjournals/joss-eics can https://www.softwareheritage.org be used for this purpose?

You're of course welcome to deposit the software with https://www.softwareheritage.org but you will also need to make an archive with somewhere that will give you a DOI (Figshare, Zenodo, Dryad). I realize this is extra work for you all but our current document-production toolchain requires a DOI for the archive.