[REVIEW]: DetecTree: Tree detection from aerial imagery in Python

whedon commented 4 years ago

Submitting author: @martibosch (Martí Bosch) Repository: https://github.com/martibosch/detectree Version: v0.3.1 Editor: @kthyng Reviewer: @JeffWalton-PSC , @rmsare Archive: 10.5281/zenodo.3908338

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/10979c12afde3ba59bcdbc7fbbb6ed2d"><img src="https://joss.theoj.org/papers/10979c12afde3ba59bcdbc7fbbb6ed2d/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/10979c12afde3ba59bcdbc7fbbb6ed2d/status.svg)](https://joss.theoj.org/papers/10979c12afde3ba59bcdbc7fbbb6ed2d)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@JeffWalton-PSC & @rmsare, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @kthyng know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @JeffWalton-PSC

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@martibosch) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @rmsare

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@martibosch) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

whedon commented 4 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @JeffWalton-PSC , @rmsare it looks like you're currently assigned to review this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.1007/3-540-59119-2_166 is OK
- 10.1023/A:1011139631724 is OK
- 10.1109/TPAMI.2004.60 is OK
- 10.1145/1653771.1653792 is OK
- 10.1109/MCSE.2011.37 is OK
- 10.7717/peerj.453 is OK
- 10.25080/Majora-7b98e3ed-013 is OK
- 10.25080/Majora-4af1f417-011 is OK
- 10.1038/s41592-019-0686-2 is OK

MISSING DOIs

- https://doi.org/10.25080/majora-92bf1922-00a may be missing for title: Data structures for statistical computing in python

INVALID DOIs

- 10.5555/1953048.2078195 is INVALID

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

martibosch commented 4 years ago

Regarding the DOIs: I have added the missing DOI for pandas as suggested by @whedon. I have also found that the invalid DOI (of scikit-learn) actually corresponds to a record in the digital ACM library rather than a DOI, nevertheless I could not find the DOI of the actual journal article. Shall I instead add the zenodo DOI for the library, i.e., 10.5281/zenodo.3696718?

martibosch commented 4 years ago

I have fixed the DOIs as suggested above in 0128e18

martibosch commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

kthyng commented 4 years ago

Hi everyone! We are officially starting JOSS back up today. We know that everyone has different circumstances with work and home life right now, so we want to figure out a way forward that will work for everyone.

@JeffWalton-PSC and @rmsare — are you both still able to review this submission? If so, what timelines would work for you?

JeffWalton-PSC commented 4 years ago

Kristen, I will be able to get to this over the next 2-3 weeks. Does that work?

From: Kristen Thyng [mailto:notifications@github.com] Sent: Wednesday, 20 May, 2020 13:17 To: openjournals/joss-reviews joss-reviews@noreply.github.com Cc: Jeff Walton jwalton@paulsmiths.edu; Mention mention@noreply.github.com Subject: Re: [openjournals/joss-reviews] [REVIEW]: DetecTree: Tree detection from aerial imagery in Python (#2172)

Hi everyone! We are officially starting JOSS back up today. We know that everyone has different circumstances with work and home life right now, so we want to figure out a way forward that will work for everyone.

@JeffWalton-PSChttps://github.com/JeffWalton-PSC and @rmsarehttps://github.com/rmsare — are you both still able to review this submission? If so, what timelines would work for you?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/openjournals/joss-reviews/issues/2172#issuecomment-631610435, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AI5VWGZ3VSRQG3J33KH5Q3DRSQGARANCNFSM4MDL4QJQ.

kthyng commented 4 years ago

Kristen, I will be able to get to this over the next 2-3 weeks. Does that work?

Absolutely! Thanks!

rmsare commented 4 years ago

@kthyng Yes, I will be able to complete this review within 3 weeks.

kthyng commented 4 years ago

@rmsare excellent!

rmsare commented 4 years ago

@martibosch @kthyng Thank you for your patience with my review! I enjoyed playing around with detectree.

This is a nice package that solves a remote sensing problem - tree detection in aerial imagery. It re-implements a published method in Python and this work is appropriately referenced. The API is clear and well documented and it uses the modern Python data science stack. The functionality seems like it will be useful for spatial, social science, and environmental science research, or even commercial applications in real estate or insurance.

My main suggestions are to provide more concrete examples of use in the paper and consider reorganizing the text so readers can get a high-level overview of detectree’s research applications. Adding a simple example to the docs would lower the barrier to entry for potential users.

Recommendation: Accept after minor revisions

Review suggestions:

[x] Improve statement of need and other small edits (#3)
[x] Add an example use case notebook (#4)
[x] Discuss related work (#5)

Full disclosure: The author reviewed a submission I made to JOSS last year, but I don't know him personally and we don't share any collaborators.

JeffWalton-PSC commented 4 years ago

@kthyng @martibosch Please accept my apologies for taking so long on this review. It is the first JOSS review I have been involved with. I have learned a great deal. Thank you for your patience.

DetecTree is a nice piece of software. I ran through all of the example notebooks in the detectree-example repository. I have made some comments in the detectree-example issue tracker. It took me quite a while to get all of the dependencies installed and working. The LAS-tools required some effort. As @rmsare has suggested, a simple end-to-end example on the main DetecTree repository would likely be helpful. The use of LIDAR as a method to collect training data is impressive, but adds a high level of complexity for a user who is just checking out the DetecTree software.

Recommendation: Accept after minor edits to the paper.

Minor edit suggestions:

[x] in Step 1 "involve manual digitizing"
[x] in Step 2 "exploited"
[x] "Another approach by Tianyang et al. [@tianyang2018single] employs"
[x] "The computation ... makes use"

@kthyng Thank you.

martibosch commented 4 years ago

Hello all,

first of all thank you @rmsare and @JeffWalton-PSC for your reviews.

I have addressed your issues in the detectree and detectree-example repositories. An end-to-end example is now available at https://github.com/martibosch/detectree-example/blob/master/notebooks/aussersihl-canopy.ipynb On the other hand, I have corrected the typos pointed out by @JeffWalton-PSC in 4c34a3.

Let me know if you have any further suggestions. Best, Martí

rmsare commented 4 years ago

Thanks! These changes address my review.

I have one last minor suggestion about the README. The statement of need is a bit long and hard to read. It could be split into two sentences. So

The target audience is researchers and practitioners in GIS that are interested in two-dimensional aspects of trees, such as their proportional abundance and spatial distribution throughout a region of study and thereafter assess important aspects of urban planning such as the provision of urban ecosystem services.

might become something like

The target audience is researchers and practitioners in GIS that are interested in two-dimensional aspects of trees, such as their proportional abundance and spatial distribution throughout a region of study. These measurements can be used to assess important aspects of urban planning such as the provision of urban ecosystem services.

martibosch commented 4 years ago

Indeed splitting the statement makes it more readable. I have amended it (both in the README and in the paper) in 003f76

kthyng commented 4 years ago

Ok all, it looks to me like @martibosch has addressed concerns from @rmsare.

@JeffWalton-PSC can you confirm if your review is complete and you recommend publication?

JeffWalton-PSC commented 4 years ago

@martibosch Thank you for making the changes. DetecTree is a nice piece of work.

@kthyng my review is complete. I recommend for publication.

kthyng commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

kthyng commented 4 years ago

@whedon check references

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.1007/3-540-59119-2_166 is OK
- 10.1023/A:1011139631724 is OK
- 10.1109/TPAMI.2004.60 is OK
- 10.1145/1653771.1653792 is OK
- 10.25080/majora-92bf1922-00a is OK
- 10.5281/zenodo.3696718 is OK
- 10.1109/MCSE.2011.37 is OK
- 10.7717/peerj.453 is OK
- 10.25080/Majora-7b98e3ed-013 is OK
- 10.25080/Majora-4af1f417-011 is OK
- 10.1038/s41592-019-0686-2 is OK

MISSING DOIs

- https://doi.org/10.1117/12.2529442 may be missing for title: An efficient framework for monitoring tree cover in an area through aerial images

INVALID DOIs

- None

martibosch commented 4 years ago

@whedon check references

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.1007/3-540-59119-2_166 is OK
- 10.1023/A:1011139631724 is OK
- 10.1109/TPAMI.2004.60 is OK
- 10.1145/1653771.1653792 is OK
- 10.25080/majora-92bf1922-00a is OK
- 10.5281/zenodo.3696718 is OK
- 10.1109/MCSE.2011.37 is OK
- 10.7717/peerj.453 is OK
- 10.25080/Majora-7b98e3ed-013 is OK
- 10.25080/Majora-4af1f417-011 is OK
- 10.1117/12.2529442 is OK
- 10.1038/s41592-019-0686-2 is OK

MISSING DOIs

- None

INVALID DOIs

- None

kthyng commented 4 years ago

Hi @martibosch, I have some small changes in PR #6. If you agree, please merge that. A few other items from the paper:

in Step 4: "...class might respectively..." Should this be might? I am not clear on the meaning of the sentence here.
"Zurich and Lausanne" are mentioned, but should this be a citation?
Can you please do a detailed check of the references to be sure they are accurate? In particular, check the capitalization of the titles and journals. You can preserve capitalization in your bibtex entries by adding {} around the words in which to preserve it.

martibosch commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

martibosch commented 4 years ago

Hello @kthyng, I have merged your PR, thank you for your corrections. Regarding your three comments:

I believe that I was taught that might and can are synonyms in this context, but I am no native English speaker, so we can use can as set in your PR.
I suppose that you refer to the part of:

New York achieve a pixel classification accuracy of 91.7%, whereas the example applications of DetecTree in Zurich and Lausanne achieve accuracies of 85.98% and 91.75% respectively

In such case, the accuracy scores are taken from GitHub repositories (i.e., detectree-example and lausanne-tree-canopy for which I do not have any proper citation. I hope to have a citation for the Lausanne tree canopy soon, since such a canopy map was produced in the context of a study of the urban heat islands which has been submitted to GMD. Therefore, should I leave it like that, with no citation? Alternatively, I can archive both repositories to Zenodo so that we can cite them in this manuscript.
All references should be properly formatted as of 1516fa6

kthyng commented 4 years ago

Hi @martibosch!

"might" has a bit of uncertainty to it usually, so "can" is better here I think. Is this change in there?
Ah, I see my misunderstanding. I read it as two people's last names instead of two places! Sorry about that. I haven't been to Switzerland before, sadly. You can choose what makes the most sense here.

Please let me know when this is all accounted for, thanks.

martibosch commented 4 years ago

Hello @kthyng ,

the "might" was changed to "can" in your PR, so since I merged it, the current version reads

Furthermore, the functionalities of DetecTree can be used through its Python API as well as through its command-line interface (CLI)
in the absence of a proper academic article to cite, I believe that we can leave the sentence as it is.

I hope that this addresses the two remarks.

kthyng commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

kthyng commented 4 years ago

I made a change in my PR for one "might" but there was another under Step 4 that I hadn't been sure about at the time. Can you change that one?

Next, can you verify the version, and then submit your code to an archive like Zenodo and report the doi here? Make sure that the title and author list on the Zenodo archive match those of your JOSS paper (you may have to alter the Zenodo metadata to do this).

martibosch commented 4 years ago

I have checked for all the occurences of "might" in the manuscript and replaced the one at Step 4 in 3ab057.

I have created a released named joss-paper for the version v0.3.1 of DetecTree and archived it in Zenodo at 10.5281/zenodo.3908338. Finally, I edited the title and author list from the Zenodo archive so that they now match those of the JOSS paper.

kthyng commented 4 years ago

@whedon set v0.3.1 as version

whedon commented 4 years ago

OK. v0.3.1 is the version.

kthyng commented 4 years ago

@whedon set 10.5281/zenodo.3908338 as archive

whedon commented 4 years ago

OK. 10.5281/zenodo.3908338 is the archive.

kthyng commented 4 years ago

@whedon accept

whedon commented 4 years ago

Attempting dry run of processing paper acceptance...

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.1007/3-540-59119-2_166 is OK
- 10.1023/A:1011139631724 is OK
- 10.1109/TPAMI.2004.60 is OK
- 10.1145/1653771.1653792 is OK
- 10.25080/majora-92bf1922-00a is OK
- 10.5281/zenodo.3696718 is OK
- 10.1109/MCSE.2011.37 is OK
- 10.7717/peerj.453 is OK
- 10.25080/Majora-7b98e3ed-013 is OK
- 10.25080/Majora-4af1f417-011 is OK
- 10.1117/12.2529442 is OK
- 10.1038/s41592-019-0686-2 is OK

MISSING DOIs

- None

INVALID DOIs

- None

whedon commented 4 years ago

:wave: @openjournals/joss-eics, this paper is ready to be accepted and published.

Check final proof :point_right: https://github.com/openjournals/joss-papers/pull/1516

If the paper PDF and Crossref deposit XML look good in https://github.com/openjournals/joss-papers/pull/1516, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

kthyng commented 4 years ago

@whedon accept deposit=true

whedon commented 4 years ago

Doing it live! Attempting automated processing of paper acceptance...

whedon commented 4 years ago

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

whedon commented 4 years ago

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/1517
Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.02172
If everything looks good, then close this review issue.
Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

kthyng commented 4 years ago

Congrats to @martibosch on your new paper!! Thanks to reviewers @JeffWalton-PSC and @rmsare — we are so grateful for your time and expertise!

whedon commented 4 years ago

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.02172/status.svg)](https://doi.org/10.21105/joss.02172)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.02172">
  <img src="https://joss.theoj.org/papers/10.21105/joss.02172/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.02172/status.svg
   :target: https://doi.org/10.21105/joss.02172

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://joss.theoj.org/reviewer-signup.html
Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

openjournals / joss-reviews