search

uDALES / u-dales

uDALES: large-eddy-simulation software for urban flow, dispersion and microclimate modelling
https://udales.github.io/u-dales
GNU General Public License v3.0
47 stars 16 forks source link

JOSS review comments on Review checklist requirements #137

Closed wimvanderbauwhede closed 3 years ago

wimvanderbauwhede commented 3 years ago

The Review checklist at https://github.com/openjournals/joss-reviews/issues/3055 has a few items that I can't tick at the moment because I can't find the info in the docs, so if you could please point me to it:

"Documentation A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?"

I could not find any section in the docs titled 'Statement of Need'

"Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?"

There are examples, but there does not seem to be a "make test" or similar build feature to perform tests on the build.

"Functionality [...] Functionality: Have the functional claims of the software been confirmed? Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)"

I could not find anything in the docs about validation of functionality or performance.

dmey commented 3 years ago

  1. Statement of need I think that this had already been addressed in https://github.com/openjournals/joss-reviews/issues/2950#issuecomment-761789660 but let us know if this not the case.

  2. Automated Tests We have a series of automated regression and compile tests that are run at every commit -- https://github.com/uDALES/u-dales/tree/master/tests

  3. Functionality I am not sure what is meant by this exactly -- I do not think there are any specific claims made that need to be checked but @tomgrylls may have more to say about this.

wimvanderbauwhede commented 3 years ago
  1. The form distinguishes between Statement of Need in the Documentation and in the Software Paper. In the paper I am fine with the way it is addressed in https://github.com/openjournals/joss-reviews/issues/3055 but apparently you need this in the docs as well
  2. Great, but I would not have found them had you not pointed it out. Could this maybe be added to the guide?
  3. The paper claims "capable of explicitly representing buildings and of modelling the full range of urban processes". I am sure you explain this in detail in your papers, but as JOSS reviewer I am expected to learn this from the documentation. If there are no specific claims about performance then that is fine of course.
dmey commented 3 years ago
  1. @tomgrylls I think this needs to be added to docs as well -- maybe by adding an overview section in the README.md that is a cutout from the current paper?
  2. @wimvanderbauwhede if you don't mind I think this is a bit out of scope -- this is linked from the development guide which is itself linked from the main README.md. Software testing is less of a end-user facing task.
  3. @tomgrylls this is something more on your side
tomgrylls commented 3 years ago

Thanks both - I have added to the documentation and created a PR (#140).

@dmey I will first request a review from you and another uDALES team member to make sure we are happy with these additions before then confirming that it satisfies the checklist as discussed above.

wimvanderbauwhede commented 3 years ago

OK, happy to close this issue.