[REVIEW]: popoff: POtential Parameter Optimisation for Force-Fields

whedon commented 3 years ago

Submitting author: !--author-handle-->@LMMorgan@arfon<!--end-editor-- Reviewers: @marshallmcdonnell, @lscalfi Archive: Pending

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/20aac788629856e4d0aba5f6940df285"><img src="https://joss.theoj.org/papers/20aac788629856e4d0aba5f6940df285/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/20aac788629856e4d0aba5f6940df285/status.svg)](https://joss.theoj.org/papers/20aac788629856e4d0aba5f6940df285)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@marshallmcdonnell & @lscalfi, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @dpsanders know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @marshallmcdonnell

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[ ] Contribution and authorship: Has the submitting author (@LMMorgan) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[ ] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[ ] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[ ] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @lscalfi

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@LMMorgan) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[ ] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
[ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @marshallmcdonnell, @lscalfi it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 3 years ago

Wordcount for paper.md is 1214

whedon commented 3 years ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1088/0022-3719/18/6/010 is OK
- 10.1021/acsaem.9b01870 is OK
- 10.1021/ja0434073 is OK
- 10.1021/am504351p is OK
- 10.1063/1.367521 is OK
- 10.1143/JPSJS.79SA.59 is OK
- 10.1021/jp984398l is OK
- 10.1021/jp506025k is OK
- 10.1039/C9CP01585J is OK
- 10.1098/rspa.1924.0082 is OK
- 10.1098/rspa.1938.0173 is OK
- 10.1039/a606455h is OK
- 10.1080/13642819608239107 is OK
- 10.1088/1361-651x/aa6ecf is OK
- 10.1088/0965-0393/25/1/014001 is OK

MISSING DOIs

- 10.1016/0022-3093(95)00355-x may be a valid DOI for title: Ab initio molecular dynamics for liquid metals
- 10.1103/physrevb.49.14251 may be a valid DOI for title: Ab initio molecular-dynamics simulation of the liquid-metal–amorphous-semiconductor transition in germanium
- 10.1016/0927-0256(96)00008-0 may be a valid DOI for title: Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set
- 10.1103/physrevb.54.11169 may be a valid DOI for title: Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set
- 10.2172/10176421 may be a valid DOI for title: Fast parallel algorithms for short-range molecular dynamics

INVALID DOIs

- None

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

whedon commented 3 years ago

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.13 s (477.3 files/s, 63030.3 lines/s)
--------------------------------------------------------------------------------
Language                      files          blank        comment           code
--------------------------------------------------------------------------------
Python                           37            556            833           2556
XML                               1              0            129           1790
TeX                               1             22              0            242
Jupyter Notebook                  5              0           1023            179
reStructuredText                  8            131             78            176
Markdown                          3             62              0            148
Bourne Again Shell                1             31              9            141
DOS Batch                         1              8              1             26
YAML                              1              1              4             18
make                              1              4              7              9
JSON                              3              0              0              3
--------------------------------------------------------------------------------
SUM:                             62            815           2084           5288
--------------------------------------------------------------------------------

Statistical information for the repository '6109e616eb4f18332f7085ab' was
gathered on 2021/09/21.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Benjamin Morgan                  1           232              0            1.48
LMMorgan                         5             8            256            1.69
Lucy Morgan                     82          9264           5716           95.73
Pezhman Zarabadi-Poo             1            24              0            0.15
Thomas Young Cluster             2           121             15            0.87
connorourke                      1             8              4            0.08

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
LMMorgan                      3           37.5          2.3                0.00
Lucy Morgan                3918           42.3         10.8                6.15
Pezhman Zarabadi-Poo         24          100.0          2.0                8.33

whedon commented 2 years ago

:wave: @lscalfi, please update us on how your review is going (this is an automated reminder).

whedon commented 2 years ago

:wave: @marshallmcdonnell, please update us on how your review is going (this is an automated reminder).

LMMorgan commented 2 years ago

Good morning. I was just wondering if there was any update on this? Many thanks.

marshallmcdonnell commented 2 years ago

@LMMorgan thank you for the reminder! I do apologize. I am still in the middle of working through my review but have had to take breaks while working through it. I will try to finish up and provide my feedback as soon as possible.

lscalfi commented 2 years ago

Dear @LMMorgan, sorry for the delay, I had a hard time installing the different packages.

The PopOff library is a python library for fitting force field parameters for Buckingham potentials, possibly using core-shell models. It is interfaced with the molecular dynamic code LAMMPS using the LAMMPS python interface. This adds a (complex) dependency to the PopOff library but frees the user from the need to build an interface to an MD code. The fitting is done with respect to DFT forces and stress tensors, obtained from VASP DFT simulations. This library is thought for application to bulk materials but could be easily applied to other systems. These are my remarks and questions on the paper, documentation and the library itself:

One of the main criticisms I have is that the functionality of the code is not enough explained and should be more detailed both in the paper and in the docs/readme. In particular, although known, the potential form of the Coulomb-Buckingham potential should be written down, and each fitting parameter clearly identified (called a, rho and c in the notebooks). Similarly, the core-shell model should be introduced more in details. The links to the corresponding LAMMPS potentials could be useful too. Moreover, the informations are somehow slitted between the paper, the various readme, the documentation and the Jupyter notebooks, which makes it difficult to quickly grasp all the functionalities of the code, for example what parameters can be fitted, and what are the 'different permutations of the Buckingham potential' considered. Also the details on the optimizer are only given in the paper but could be added or centralized in the docs.
The paper mentions other software to fit interatomic potentials, could more details on their functionalities be included? Also, why only Buckingham potentials can be fitted? It seems it would be easy to generalize this to other potentials implemented in LAMMPS.
The readme file should be completed: it only contains instructions to build the LAMMPS python library. It should also give instructions to build the PopOff package and it would be useful to have a few lines on how to find and build the docs. In the docs, it would be easier if the installation instructions for the LAMMPS python library and PopOff were given together (instead of in the Dependencies).
Which LAMMPS packages are needed for PopOff and for testing purposes? The packages 'snap' and 'user-reaxc' couldn't be installed in my LAMMPS (Package does not exist error) but this didn't cause problems for running the tests.
The documentation and readme uses 'I', this should probably be updated.
There is a section for asking help at the end of the docs but no community guidelines for developers (merge requests allowed?...). These informations should be added.
Could you cite works where this package has been used? Please also specify the mention to 'a group at Newcastle University'.
The sentence in the paper 'This can be changed in fitting_code.py, however, this is not given as a suggested change in the manual.' is confusing. Should these changes be suggested? Are they hard-coded in the package?
Regarding the Jupyter notebooks, I was unable to fully run them. Indeed, the example given is Na3OCl, but the notebooks are written for LiNiO2. I tried running the script 'userguides/example_Na3OCl/output/potential_fitting.py' but it is apparently an old version. Indeed, it calls the library BuckFit (I guess an old version that should maybe be mentioned somewhere). When changed to PopOff, I have a weird FileNotFoundError in lammps_data.py, line 252, when trying to open the 'lammps/coords1.lmp' file for writing. Please provide either the files for LiNiO2 or change the notebooks to fit to the NaO3Cl example. Nevertheless, I already have some questions:
Why the core-shell models should have two spring constants k1 and k2?
Only the parameters for Li-O, Ni-O and O-O are given, do you have mixing rules for the others or how can they be set?
What is the meaning of the number of fits? Is this meant for running the same procedure on a different set of structures? Or with different random seeds?
In Training Set and Fitting, the directory is called 'vasprun' but should be 'vaspruns'. Also in the 'random_set_of_structures' function, if the optional parameter 'seed' is omitted then the function returns an empty list. Later, in the section 'Randomly selecting sets of structures', it is said that the default is None, while it is False.

Moreover, there are some details to be fixed:

The name of the package is not consistent through the paper, the repository and the documentation. popoff, PopOff and POPOFF have been used. Please use the same name everywhere (also in the paths in the documentation for example).
In the paper, give the meaning of the abbreviation NMC.
Overall there are several orthographic mistakes (in the paper: 'fittted', in the readmes and the docs: 'potenial', 'relivant', 'ca be found in', in the notebook: 'paralising', 'agian', 'corespond', 'trianing', 'creat').
The 'Module contents' section of the PopOff submodules in the docs is empty, and the link to 'Training set guide' for VASP doesn't work.
The command to unzip the vasp files in the tests directory should be gzip -d <file>. It would also be interesting to briefly explain what is tested and how.

@dpsanders I think I didn't click on the acceptation link on time and cannot modify the above todo list, would it be possible to change this? Thank you

dpsanders commented 2 years ago

👋 @lscalfi: Many thanks for your detailed review. I'll re-invite you

dpsanders commented 2 years ago

@whedon commands

whedon commented 2 years ago

Here are some things you can ask me to do:

# List all of Whedon's capabilities
@whedon commands

# Assign a GitHub user as the sole reviewer of this submission
@whedon assign @username as reviewer

# Add a GitHub user to the reviewers of this submission
@whedon add @username as reviewer

# Re-invite a reviewer (if they can't update checklists)
@whedon re-invite @username as reviewer

# Remove a GitHub user from the reviewers of this submission
@whedon remove @username as reviewer

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

# Change editorial assignment
@whedon assign @username as editor

# Set the software archive DOI at the top of the issue e.g.
@whedon set 10.0000/zenodo.00000 as archive

# Set the software version at the top of the issue e.g.
@whedon set v1.0.1 as version

# Open the review issue
@whedon start review

EDITORIAL TASKS

# All commands can be run on a non-default branch, to do this pass a custom 
# branch name by following the command with `from branch custom-branch-name`.
# For example:

# Compile the paper
@whedon generate pdf

# Compile the paper from alternative branch
@whedon generate pdf from branch custom-branch-name

# Remind an author or reviewer to return to a review after a
# certain period of time (supported units days and weeks)
@whedon remind @reviewer in 2 weeks

# Ask Whedon to do a dry run of accepting the paper and depositing with Crossref
@whedon recommend-accept

# Ask Whedon to check the references for missing DOIs
@whedon check references

# Ask Whedon to check repository statistics for the submitted software
@whedon check repository

EiC TASKS

# Invite an editor to edit a submission (sending them an email)
@whedon invite @editor as editor

# Reject a paper
@whedon reject

# Withdraw a paper
@whedon withdraw

# Ask Whedon to actually accept the paper and deposit with Crossref
@whedon accept deposit=true

dpsanders commented 2 years ago

@whedon re-invite @lscalfi as reviewer

whedon commented 2 years ago

OK, the reviewer has been re-invited.

@lscalfi please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations