[REVIEW]: EmissV: An R package to create vehicular and other emissions by Top-down methods to air quality models

whedon commented 6 years ago

Submitting author: @Schuch666 (Daniel Schuch) Repository: https://github.com/atmoschem/EmissV Version: 0.664.3 Editor: @arfon Reviewer: @nuest, @benmarwick Archive: 10.5281/zenodo.1451027

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/071d027997ac93d8992099cb5010a044"><img src="http://joss.theoj.org/papers/071d027997ac93d8992099cb5010a044/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/071d027997ac93d8992099cb5010a044/status.svg)](http://joss.theoj.org/papers/071d027997ac93d8992099cb5010a044)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@nuest, @benmarwick, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @arfon know.

Review checklist for @nuest

Conflict of interest

[x] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Version: Does the release version given match the GitHub release (0.664.3)?
[x] Authorship: Has the submitting author (@Schuch666) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Authors: Does the paper.md file include a list of authors with their affiliations?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @benmarwick

Conflict of interest

[ ] As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

[ ] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[ ] Repository: Is the source code for this software available at the repository url?
[ ] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[ ] Version: Does the release version given match the GitHub release (0.664.3)?
[ ] Authorship: Has the submitting author (@Schuch666) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[ ] Authors: Does the paper.md file include a list of authors with their affiliations?
[ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[ ] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

whedon commented 6 years ago

Hello human, I'm @whedon. I'm here to help you with some common editorial tasks. @nuest it looks like you're currently assigned as the reviewer for this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

whedon commented 6 years ago

Attempting PDF compilation. Reticulating splines etc...

whedon commented 6 years ago

--> Check article proof :page_facing_up: <--

nuest commented 6 years ago

Sorry, I'm at a conference this week. Will complete this next week!

arfon commented 6 years ago

@nuest @benmarwick - friendly reminder to complete your reviews when you get a chance :grin:

benmarwick commented 6 years ago

Thanks, working on it!

Schuch666 commented 6 years ago

Hey guys!

Do not forget me

This is a kind reminder :)

arfon commented 6 years ago

@nuest @benmarwick - friendly reminder to complete your reviews when you get a chance 😁

Yes, please take a look at this when you get a chance please!

nuest commented 6 years ago

Sincere apologies for the delay, too many GH emails and this one fell through the cracks. I started my review and have some first comments that can be addressed rightaway, see issues linked above. I'll start the functional evaluation next.

nuest commented 6 years ago

I gave the paper.md a first read, some comments/questions:

Why are the emission inventories the scapegoat? Why not the athmospheric data?
Have you tested the package with data other than for Sao Paulo State?
I recommend not to use abbreviations in the example code for object names ("SP", "MG"), makes it harder to follow.
The file sample.tiff is missing, probably must be dmsp.tiff
The logging output from areaSource differs on my machine compared to what the paper has, starting at the fourth decimal place... is this relevant or expected?
The scale of the created plot differs for Fig. 1:

The code to create the second figure is missing.
The final command fails on my machine:

> CO_emissions <- emission(TOTAL,"CO",list(SP = SP, MG = MG),grid,mm=28, plot = T)
[1] "calculating emissions for CO using molar mass = 28 ..."
Error in seq.default(zrng[1], zrng[2], length.out = cuts + 2) : 
  'from' must be a finite number
In addition: Warning messages:
1: In asMethod(object) :
  complete map seems to be NA's -- no selection was made
2: In min(x) : no non-missing arguments to min; returning Inf
3: In max(x) : no non-missing arguments to max; returning -Inf

References
- Eixport has a JOSS paper with a DOI, which should be used for the citation
- Pulles & Heslinga also has a DOI: doi.org/10.13140/RG.2.1.2082.8007
- Why do you cite ncdf4, but not other packages (raster, sp?)

In my opinion the submission needs some work. I am also not a domain expert, so I cannot say anything about the validity of the method included in the package.

@benmarwick Are you familiar with the methodology and kind of analysis the package performs?

benmarwick commented 6 years ago

Yes, to start with, I'd love to see another attempt at a clear statement of what problems the software is designed to solve and who the target audience is? Who are the users likely to be? I've just done a tiny bit work helping students in atmospheric chemistry, so the methods here are not familiar to me.

I can run the example code until here:

TOTAL  <- totalEmission(veiculos,EF,pol = c("CO"),verbose = T)
Error: $ operator is invalid for atomic vectors 
5.
as.character.units(structure(1127549.32468524, units = structure(list(
    numerator = "t", denominator = "year"), .Names = c("numerator", 
"denominator"), class = "symbolic_units"), class = "units")) 
4.
as.character(structure(1127549.32468524, units = structure(list(
    numerator = "t", denominator = "year"), .Names = c("numerator", 
"denominator"), class = "symbolic_units"), class = "units")) 
3.
paste("Total of", pol[i], ":", sum(total_t_y), units::deparse_unit(total_t_y)) 
2.
print(paste("Total of", pol[i], ":", sum(total_t_y), units::deparse_unit(total_t_y))) at totalEmission.R#62
1.
totalEmission(veiculos, EF, pol = c("CO"), verbose = T)

I've noted this at the repo here: https://github.com/atmoschem/EmissV/issues/16

Update: seems I was missing some Linux libraries. I'll continue testing on Windows.

Schuch666 commented 6 years ago

Thank @nuest and @benmarwick

I'm working on the text (and the code). Soon I will update the package and write here some detailed explanations

Schuch666 commented 6 years ago

Hi @nuest and @benmarwick, thanks for the suggestions, some corrections have already been made to the package and the suggestions in the article have already been incorporated.

@nuest I make some changes in the paper (and in the repository)

added to the paper a reason to emission inventories being the scapegoat of air quality model input
The package was tested for some cases in São Paulo and some Brazil adjacent states, for now, the main reason is that São Paulo has a Pollution problem and an available air quality monitory network to compare the results
I changed the abbreviation for states names
that's right, the file sample.tiff was changed for dmsp.tiff, this is a smaller file (for CRAN submission).
The differences of area source are mainly because of the difference of resolution of the image and the loss of accuracy, it is not a problem (but is an updated version).
The figure 1 shows g/d on each image pixel.
The second figure is created in the emission function with plot = T, the reason is that output is a matrix with can be written in model input files (with the cited packages) and is not georeferenced
About the error: the function rasterSource was updated (to always deal with NAs)
The References was added to paper, I cited the packages that can be written in netcdf format files, eixport has specific functions for write in wrf input files

@benmarwick the main problem is the need of input data for air quality models, depending on the region there is no information about the temporal or spatial distribution of the vehicular emissions or in some cases, the information is at a national level and not updated for decades. The audience is students and researchers who are interesting in using numerical air quality models but don't have a good inventory or wants an alternative inventory.

The function vehicles have more detailed description and the number of vehicles is integer now.

I'm Working on the package remain Issues!

Schuch666 commented 6 years ago

Hi @nuest , @benmarwick and @arfon

I have some news: EmissV was accepted on CRAN.

The paper was revised:

added more detailed description of the problem and the package
the example was slightly reformulated (the dmsp_hi-res.tiff was added to the repository and example)
added references
grammatical and orthographic corrections

When you have time, take a look.

thank you

PS: We are planning the use of EmissV in some Argentine cities in the next months.

arfon commented 6 years ago

I have some news: EmissV was accepted on CRAN.

Great! Thanks for the update @Schuch666.

@nuest, @benmarwick - could you come and take another look at this sometime in the next couple of weeks?

Schuch666 commented 6 years ago

news:

Automated tests added to the package

arfon commented 6 years ago

Ping @nuest, @benmarwick :smile: please come any take another look at this package sometime soon.

Schuch666 commented 6 years ago

Hey guys @nuest @benmarwick

Please come any take another look at this package

This is a kindly reminder

ibarraespinosa commented 6 years ago

hi @arfon, I'm co-author of this paper and package. Is it possible to speed up the review process? Thanks! Sergio

arfon commented 6 years ago

Friendly reminder to come back to this review sometime soon please @nuest & @benmarwick

nuest commented 6 years ago

@Schuch666 Thanks for taking the time to work on our suggestions, and for your understanding that this takes longer than it should. Congrats for getting the package on CRAN!

@whedon generate pdf

nuest commented 6 years ago

Ok, so PDF re-generation does not seem to work... @arfon did I miss something here?

Don't want to be too picky on this, but for me the reason to make emission the "scapegoat" must be connected to the other measured variables somehow. Is it the case that athmosphere is not blamed because these values are easier to measure? Is the landscape not blamed because it is not changing? This might be obvious for you, but IMO the JOSS abstract should be understandable for most people.
I can now compile a PDF from paper.md without any issues - good!

I get slightly different results in some outputs - is that an issue? See e.g.

> TOTAL  <- totalEmission(fleet,EF,pol = c("CO"),verbose = T)
Total of CO : 1128297.0993334 t year-1

regarding the differences in the outputs - would you consider updating the paper with a short statement (or the function docs) that the outputs might differ slightly because of... X
I'd like to restate Ben's point about the statement of need. AFAICS the first paragraph is almost there. Do you want to help to better understand anthropogenic emissions? Do you want to stop the emissions being used as a scapegoat? What would a user want to... "estimates vehicular emissions by a top-down approach". This is probably totally clear to you, but the first paragraph can just as well welcome any audience.

@arfon Does the author have to make a new release on GitHub? Not sure how to answer the "matches" checklist item. Thanks!

arfon commented 6 years ago

@whedon generate pdf

whedon commented 6 years ago

Attempting PDF compilation. Reticulating splines etc...

whedon commented 6 years ago

--> Check article proof :page_facing_up: <--

arfon commented 6 years ago

Ok, so PDF re-generation does not seem to work... @arfon did I miss something here?

☝️ try @whedon generate pdf

@arfon Does the author have to make a new release on GitHub? Not sure how to answer the "matches" checklist item. Thanks!

Basically this is a question about the version number you see here (0.664.3) and what you find on GitHub.

arfon commented 6 years ago

Hi @benmarwick - do you know when you will be able to complete your review by?

nuest commented 5 years ago

I see a few minor questions and some need for rephrasing, as described in https://github.com/openjournals/joss-reviews/issues/662#issuecomment-411080315, other than that I'm done.

Schuch666 commented 5 years ago

Hey @nuest , thanks for the suggestions.

The reason itself of emission being the "scapegoat" is that a difficult and highly multidisciplinary task. In the case of the anthropogenic emissions, the main difficulty is to create an updated inventory (sometimes there is no inventory available) with the available information (sometimes there is a very low amount of information available). I think one reason can be that the responsibility for the emissions is not of the model user in many cases.

I added some text to the Introduction:

'The anthropogenic emissions, especially vehicular emissions, are highly dependent on human activity and constantly changing due to various factors ranging from economic (such as the state of conservation of the fleet, renewal of the fleet and the price of fuel) to legal aspects (such as the vehicle routing).'

Atmospheric models are continuously developed by research institutions. As well as ways to assimilate a wide variety of data (atmospheric, terrestrial, vegetation, etc.) as satellite images, classifications and parameterizations. The EmissV allow the user to create the emissions with data like tables, shapefiles and images this give the user some power to use available information and more responsibility.

A statement of need of the package use has added next to this:

'The EmissV is an R package that estimates vehicular emissions by a top-down approach, the emissions are calculated using the statistical description of the fleet at available level (National, State, City, etc). The following steps show an example of the workflow for calculating vehicular emissions, this emissions are initially temporally and spatially disaggregated, and then distributed spatially and temporally to be used as input in numeric air quality models such WRF-Chem [@Grelletal2005].'

The values slightly different results in some outputs had updated, the reason it can be that the function vehicles operate on integer numbers only now (that make more sense to me).

If any point can be improved let me know

PS: I changed the 'veiculos' for fleet

Schuch666 commented 5 years ago

Hi, @benmarwick , are you still with us?

nuest commented 5 years ago

@Schuch666 Please double check the documentation to mention that output values might differ slightly. If users run your algorithms and compare outcomes (e.g. in a reproducible research set-up, a reviewer runs an article manuscript) they should be aware that small differences can occur and why.

That's all from me then: LGTM 👍

arfon commented 5 years ago

@Schuch666 Please double check the documentation to mention that output values might differ slightly. If users run your algorithms and compare outcomes (e.g. in a reproducible research set-up, a reviewer runs an article manuscript) they should be aware that small differences can occur and why.

Once you've addressed this comment @Schuch666 I think we're good to accept here.

Schuch666 commented 5 years ago

thank you @arfon

I put a note in the text talking about these differences that can appear in the calculated values and in the figures due to changes between versions of the package (in the code or in the sample files) and also of the version of its dependencies.

Schuch666 commented 5 years ago

@whedon generate pdf

whedon commented 5 years ago

Attempting PDF compilation. Reticulating splines etc...

whedon commented 5 years ago

:point_right: Check article proof :page_facing_up: :point_left:

arfon commented 5 years ago

@Schuch666 - At this point could you make an archive of the reviewed software in Zenodo/figshare/other service and update this thread with the DOI of the archive? I can then move forward with accepting the submission.

Schuch666 commented 5 years ago

Hi @arfon

I upload to Zenodo with the following DOI: 10.5281/zenodo.1451027

best regards

arfon commented 5 years ago

@whedon set 10.5281/zenodo.1451027 as archive

whedon commented 5 years ago

OK. 10.5281/zenodo.1451027 is the archive.

arfon commented 5 years ago

@nuest - many thanks for your review here ✨

@Schuch666 - your paper is now accepted into JOSS and your DOI is https://doi.org/10.21105/joss.00662 :zap: :rocket: :boom:

whedon commented 5 years ago

:tada::tada::tada: Congratulations on your paper acceptance! :tada::tada::tada:

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](http://joss.theoj.org/papers/10.21105/joss.00662/status.svg)](https://doi.org/10.21105/joss.00662)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.00662">
  <img src="http://joss.theoj.org/papers/10.21105/joss.00662/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: http://joss.theoj.org/papers/10.21105/joss.00662/status.svg
   :target: https://doi.org/10.21105/joss.00662

This is how it will look in your documentation:

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: http://joss.theoj.org/reviewer-signup.html
Making a small donation to support our running costs here: https://www.flipcause.com/secure/cause_pdetails/Mjk3Nzk=

openjournals / joss-reviews