editorialbot commented 2 years ago

Submitting author: !--author-handle-->@jranalli@jgostick<!--end-editor-- Reviewers: @espottesmith, @fwitte Archive: 10.5281/zenodo.7262173

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/42d6ea5759b5467dad54709db33ea03d"><img src="https://joss.theoj.org/papers/42d6ea5759b5467dad54709db33ea03d/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/42d6ea5759b5467dad54709db33ea03d/status.svg)](https://joss.theoj.org/papers/42d6ea5759b5467dad54709db33ea03d)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@espottesmith & @fwitte, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @jgostick know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @espottesmith

editorialbot commented 2 years ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot commented 2 years ago

Software report:

github.com/AlDanial/cloc v 1.88  T=0.33 s (151.5 files/s, 63758.5 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          22           2511           6128           7678
HTML                            13            160              0           2804
TeX                              8            257              5            933
Markdown                         3             61              0            256
CSS                              1             25              5            150
XML                              1              2              0             35
YAML                             1              1              4             18
Bourne Shell                     1              3              1              4
-------------------------------------------------------------------------------
SUM:                            50           3020           6143          11878
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

editorialbot commented 2 years ago

Wordcount for paper.md is 1471

editorialbot commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1109/FIE.2016.7757589 is OK
- 10.1021/ie4033999 is OK

MISSING DOIs

- None

INVALID DOIs

- https://peer.asee.org/28757 is INVALID because of 'https://doi.org/' prefix
- 10.18260/1-232083 is INVALID

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

espottesmith commented 2 years ago

Review checklist for @espottesmith

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/chmarti1/PYroMat?
[ ] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@jranalli) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

espottesmith commented 2 years ago

My review is complete.

The authors have produced and described pyromat, a utility for calculation of gas-phase (and, to a certain extent, mixed gas/liquid) thermodynamic properties, based on established models. They provide data for ~1000 unique species, which include both common and uncommon gas-phase molecules and ions. The code is written clearly, and the documentation is, for the most part, quite good. While the use case for pyromat in research seems unclear to me (note: I do not study gas-phase or combustion thermochemistry), this seems an extremely necessary tool for students of chemistry/chemical engineering, as well as industrial engineers. My comments are below.

Compliments:

It seems that, provided the user has some data available, this code is (relatively) easily extensible. It also appears to be relatively easily configurable, though better documentation on all of the configuration parameters could perhaps be desired.

Minor issues:

The LICENSE.txt file refers to the GPL but does not reproduce the text of the license. As I understand JOSS' guidelines, this is not acceptable. I have created an issue (https://github.com/chmarti1/PYroMat/issues/54) regarding this.
One of the examples in the README does not work, though it does with small modifications (see https://github.com/chmarti1/PYroMat/issues/55)
In the text of the paper, references should probably be made to the models used to populate the data in pyromat.
There are no guidelines on how an external developer should contribute to the code.

Major issues:

There are areas where the code feels incomplete. Most critically, there is not a test suite. Validation - ensuring that you obtain expected values for a small set of materials, or ensuring that the predicted values are close to experimental values for well-studied substances like water - is essential to give the user confidence in the reported results.
Some features also more generally feel half-finished; in apps/cycle.py, for instance, the RefrigerationCycle, OttoCycle, and DieselCycle classes are defined but not implemented.

jranalli commented 2 years ago

Thank you very much for your time in the review. I'd just like to clarify the two major issues:

We have some tests stored in branch tests-jranalli. Specifically the files test_ig and test_mp1. Is there some type of documentation of that we could add that would help to satisfy this concern?
I agree that the cycle codes are half finished. It represents a longer term goal for the project to include some more holistic calculations/educational modules. The part of the code that we're really submitting is the primary property calculation portion. Is there a way that documenting that would satisfy the concern, or would it be necessary to remove those from the library.

jgostick commented 2 years ago

Hi @espottesmith, great job on the review! Your efforts and haste are really appreciated.

@jranalli, regarding the comments of @espottesmith...

I think that given what your package is about (computing thermodynamic properties) you really should have a visible and active test suite. Getting github actions to run pytest on your code for each and every commit is easy to setup now. Since you have tests already written then you just need to do some minor configuration stuff.
Software is a work in progress and it is expect that things be added. I would suggest that you not put a wish list of future enhancements in the JOSS paper since you may never actually get them done. Just focus on what the package actually does at the moment. If these cylce calculations are not functional yet, then consider just removing them for this release, then you can add them to the package when the time comes.

BTW, the 2nd reviewer requested a few week delay in starting since they were on vacay. I expect they'll be dropping by to add their review shortly, so hopefully you can work on the current suggestions.

chmarti1 commented 2 years ago

Thanks very much for your work. I'll make a few comments here:

With Respect to Test Suites This idea really needs to be divided into two parts: (1) the testing required to validate a new data set that is imported into the suite, and (2) testing required to validate new class functionality or new features.

(1) New data are carefully validated against reference data provided by the same sources from which the models are entered. This is a time-consuming process - for multi-phase source data, it has to be done manually for every data source, since the raw data are being pulled from publications. However, once validated, the work is complete, and does not need to be repeated with new releases, so no automation suite is used.

We deliberately do not currently publish validation data. In older versions of the code, I used to include records of my validation work, but I have received emails from private companies that were attempting to interpret this as a warranty of some kind. My official policy is that the sources of the original data are cited, and users should always validate their calculations against their own experimental data in the context of their application.

However, the codes that were used to validate the original ideal gas model data are actually still coded directly into the classes. The ig class has a method called _ig__test(), and ig2 has _test(). The ig substances even have validation data tables embedded directly in their raw data. _ig__test() now complains about errors because it has not been updated along with major version changes, but it still successfully validates the original source data. This has even been used to demonstrate continuity errors in NIST's source data.

(2) On this front, @jranalli deserves credit for making similar arguments for some time. In brief, this is a proposal that makes sense, but to implement it well, it would require an effort at least as difficult as writing the package itself. Now that @jranalli is contributing so heavily to the project, it may make sense to include his validation suites in the main branch.

The challenge is a familiar one: the number of permutations of input/output possibilities is now so large (e.g. array/scalar, liquid/mixed/vapor/super-critical, primary/inverse properties, near limits/centered, countless permutations of units, and all of the combinations of those), that historically, hard-coded test suites have not performed well enough to justify the effort. So far, our most successful model has been to run ad-hoc targeted tests to try to break the new functionality, then @jranalli tries harder with his own test suite, then we release to the community. This workflow has functioned pretty well so far, but we are always happy to improve.

Undocumented Partially Coded Modules True - we do have some undocumented code in the master branch. It is not currently part of package functionality. The apps module is utterly undocumented (except in-line) and is not imported by default. That module actually serves a specific (if slightly secret) purpose for our efforts to port to a web-based interface for PYroMat. The code there is part of a (still live) proof-of-concept portion of the website. It will be cleaned up in later releases, but we have decided to leave it as an easter egg for any interested users who may want to do their own developing.

LICENSE True. That will be easily corrected.

README failure This is actually a change in the functionality of info() in the last release. The new behavior was documented on the website, and in the handbook (see sections 2.2.3 and 2.2.4), but I forgot to update the README. Oops. Good catch!

jranalli commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

jranalli commented 2 years ago

Fixes implemented in the repository:

License file extended
README demo corrected
Continuous Integration for functionality tests added
Moved unfinished code (i.e. apps folder) to a separate branch to eliminate confusion.
Citations added to mention of models in text of the paper (see previous post for updated PDF)

fwitte commented 2 years ago

Quick update from me @jgostick, @jranalli: I will start working on the review this week and hope to finish it on the weekend.

jranalli commented 2 years ago

Updates to the text of the paper based on comments from @fwitte. Rebuilding

jranalli commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

fwitte commented 2 years ago

Thanks for already answering my initial comments, I still have some small parts to do, which I could not finish to work on last weekend. Once I have done that, I will give my review here.

jgostick commented 2 years ago

Hey all, how are things progressing here?

fwitte commented 2 years ago

Thank you for reminding me @jgostick and thank you for your patience @jranalli. My schedule was quite packed, I want to check two more features of the package (mixtures and the water IF97) to finalize. I will let you know here after that. I hope I can fit that into my weekend.

chmarti1 commented 2 years ago

FYI, the if97 class is deprecated. The mp1 class is now used by all multi-phase substances (including water).

fwitte commented 2 years ago

But it uses the IF97 equations, right? I was using the mp1 class and understood it was an implementation of the IF97 equations.

chmarti1 commented 2 years ago

Good question. mp1 uses a generalized "Span and Wagner" equation of state that applies over both liquid and vapor phases without piece-wise transitions. If you are curious, the mp1 model is developed in some detail in chapter 7 (pg 97-114) of the PYroMat handbook.

The if97 class uses the IF97 piece-wise formulation that you probably know already. The IAPWS actually published an earlier model in 1995 using the Span and Wagner formulation, and it was updated in 2018.

The IF97 is handy for engineering use, because it includes polynomial expansions good for "inverse" property calculations, so it was the first thing I tried. If I had to write a code to model a Rankine cycle from scratch, I'd use IF97. However, Span and Wagner equations of state are now almost universal in applications like PYroMat because:

Piece-wise approaches are heavily dependent on the substance being modeled. The regions' number, their arrangement, and even their formulation is so variable, it makes writing a generalized class a daunting task.
Piece-wise approaches require the algorithm to determine the region of the state prior before calculating properties. If the primary properties are known, that is simple, but in any other case, even determining the region can require iteration before any properties are calculated.
Numerous models using the Span and Wagner approach are now available in the literature, so extensibility to multiple substances is not a problem.
For super-critical, vapor, and mixture states, (temperature, density) is a numerically superior means for determining the state. It addresses almost all of the singularities that occur near phase changes and the critical point, and it even permits the calculation of meta-stable properties. PYroMat does not currently support meta-stable states, but the underlying models are theoretically capable of estimating their properties.
Once the potential numerical convergence problems are addressed, the numerical iteration required to specify states by (T,p) when the properties are calculated from (T,d) is not very expensive. Really, this problem has to be solved with IF97 anyway, because it uses (T,d) in the super-critical region.

fwitte commented 2 years ago

@chmarti1, thank you very much for the detailed explanation. I was wondering, why I could not replicate some of the bugs in IF97, which seem to exist in both, CoolProp and the IAPWS package, also see https://github.com/CoolProp/CoolProp/issues/1918.

fwitte commented 2 years ago

@jgostick: Just wanted to add my checklist, do you know, where it has gone?

Nevermind, copy pasted the other one.

fwitte commented 2 years ago

Review checklist for @fwitte

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/chmarti1/PYroMat?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@jranalli) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

fwitte commented 2 years ago

@jranalli, @chmarti1: Thank you for your replies and in depth explanation. Here are a few lines for my review:

Summary

Overall, I find this a very useful package. To my knowledge, the NASA polynomes are not included in other open source packages like thermo, CoolProp and iapws, but only in EES or other proprietary software. Especially the extensive handbook is well written and contains a lot of information helping both, people learning thermodynamics and those, who have been working with unit based simulation but need a refresher in fluid property modeling.

The online documentation is also well written and features some small gimmicks with the live apps, giving potential users a quick idea, what the software does. I have some suggestions for the online documentation, which I listed under ideas/suggestions, which are not mandatory for the paper to be accepted.

Major Issues

All major issues have been fixed already, see the comments of @espottesmith

Minor Issues

Ideas and Suggestions for future development

Make an API to let the user build his/her own mixtures: While you can just copy a .hpd file, give it a new id and some mixture contents, this might be hard to find for less experienced users and you are limited to a somewhat fixed mixture composition. It would be cool, if the user could build his or her own mixtures by providing the molar or mass based mixture contents and it does not seem too hard to implement.
I often prefer an non live version of the help, which I can search through while developing code. In my opinion, it is very helpful if classes and and methods are written somewhere for people who want to include your package within their software. There are currently some ideas for reworking the fluid property back end of my main package, and I am very much considering adding your package as option there.
Finally, a search function letting users search the online documentation for api, examples, explanations, ... would be a good feature. (I have about 100 searches per month on my main package.)

I am not sure, how complicated it is to include the last two features in your documentation. I do very much like using sphinx (with readthedocs if you do not want to self-host) as it is very easy and convienient to include these.

Conclusion

Recommend accepting the paper as is after the last two issues have been resolved.

jranalli commented 2 years ago

Thanks for your work on the review.

Added a pull request for the extra_requires
Discussing the guidelines for contribution with @chmarti1

Definitely appreciate the suggestions and we'll consider those as we move forward.

jranalli commented 2 years ago

Thanks again to reviewers for you time! As of now, pull requests have been implemented regarding all the major and minor issues that had been raised by reviewers. Not sure how the process continues, but if anything else is needed from us, please just speak up!

jgostick commented 2 years ago

Great stuff everyone...the above review thread shows a very comprehensive review and revision process indeed. I have confirmed that via the discussion and various merged PRs that all of the reviewers comments and concerns have been addressed. So I will move forward with accepting it. There few housekeeping items to get to first though. I will paste a checklist below.

jgostick commented 2 years ago

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

[x] Double check authors and affiliations (including ORCIDs)
[x] Make a release of the software with the latest changes from the review and post the version number here. This is the version that will be used in the JOSS paper.
[x] Archive the release on Zenodo/figshare/etc and post the DOI here.
[x] Make sure that the title and author list (including ORCIDs) in the archive match those in the JOSS paper.

Editor Tasks Prior to Acceptance

[x] Set article DOI with @editorialbot set <DOI here> as archive
[x] Set version with @editorialbot set <version here> as version
[x] Double check rendering of paper with @editorialbot generate pdf
[x] Specifically check the references with @editorialbot check references and ask author(s) to update as needed
[x] Do a 'dry run' of acceptance with @editorialbot recommend-accept

jranalli commented 2 years ago

Ok so I don't know how to mark the checkboxes, but I have double checked that everybody's ORCID and affiliation appears correctly on the paper, the version release number is 2.2.4 and it is uploaded to Zenodo with a DOI of https://doi.org/10.5281/zenodo.7262173

jgostick commented 2 years ago

@editorialbot set 2.2.4 as version

editorialbot commented 2 years ago

Done! version is now 2.2.4

jgostick commented 2 years ago

@editorialbot set 10.5281/zenodo.7262173 as archive

editorialbot commented 2 years ago

Done! Archive is now 10.5281/zenodo.7262173

jgostick commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

jranalli commented 2 years ago

Authorship and affiliations all match with expectations. The title on the proof is correct. ORCIDs are all correct as well.

jgostick commented 2 years ago

@jranalli is there any reason why Jacob Moore is on the paper but not on the repo or listed in the zenodo archive? Also, we like to have orcids on the zenodo archive as well...can you see about linking these?

jranalli commented 2 years ago

He has been a contributor to the overall project for years but hasn't made code contributions and so doesn't show on Github. We looked at manually adding an author to the Zenodo archive, but it didn't look like there was an easy way to do so. We can take a look at whether there's a different method to do so, unless you have some suggestion that we didn't think of?

danielskatz commented 2 years ago

It's very easy to add an author to Zenodo. See the section labeled "How can I edit the metadata of a published record?" in https://help.zenodo.org for how to do this.

fwitte commented 2 years ago

@jranalli: Usually, you can go to zenodo, click on edit publication (only the person who uploaded that publication) and then enter the names of the researches. Looks like so:

grafik

jranalli commented 2 years ago

@chmarti1 created the archive, so I'll ask him to take a look.

jgostick commented 2 years ago

@editorialbot check references

editorialbot commented 2 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1109/FIE.2016.7757589 is OK
- 10.1021/ie4033999 is OK
- 10.1038/s41586-020-2649-2 is OK

MISSING DOIs

- None

INVALID DOIs

- https://peer.asee.org/28757 is INVALID because of 'https://doi.org/' prefix
- 10.18260/1-232083 is INVALID

jgostick commented 2 years ago

Hi @jranalli, the references in the pdf need some work. In addition to the issues pointed out above, there are many references missing their doi. 3 of the last 4 in the list have dois at the bottom of the page: https://doi.org/10.1023/A:1022362231796. Not sure about the NASA one, but can you take a look?

jgostick commented 2 years ago

Also, for the nist paper, what about: https://doi.org/10.1021/acs.iecr.2c01427 instead of just linking to the nist website

jranalli commented 2 years ago

@editorialbot generate pdf

editorialbot commented 2 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

openjournals / joss-reviews

[REVIEW]: PYroMat: A Python package for thermodynamic properties #4757

Status

Reviewer instructions & questions

Checklists

Review checklist for @espottesmith

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @fwitte

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Summary

Major Issues

Minor Issues

Ideas and Suggestions for future development

Conclusion

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

Editor Tasks Prior to Acceptance