materialsproject / custodian

A simple, robust and flexible just-in-time job management framework in Python.
MIT License
139 stars 109 forks source link

Feature request: Nonsensical/non-existent parameter check for INCAR files #100

Open CifLord opened 5 years ago

CifLord commented 5 years ago

VASP by default does not have a way of checking if the user entered in the name of an INCAR parameter correctly and therefore will still run even if there are nonsensical parameters in the file. e.g. If one were to set a value for "NBAND" instead of "NBANDS" in the INCAR, VASP will ignore any the former. Another example is if someone sets METAGGA=Scan in ver. 5.4.1 even though this metagga parameter does not exist in this version, nothing will happen. This can make troubleshooting confusing sometimes if a user gets results that are not aligned with their expectations and don't know why.

I don't think custodian has such checks yet, but would it be possible to implement it?

shyuep commented 5 years ago

While I would like this to be implemented, I frankly have no visibility into all possible parameters in VASP and have no influence over future keywords. If VASP releases a list of valid keywords for every single VASP version, we can check it in some way. But I don't think such a list exists.

mkhorton commented 5 years ago

Actually I think we might have a route to get all possible VASP parameters; aren't they enumerated in the vasprun.xml file?

mkhorton commented 5 years ago

We could only do this for the current version though, unless anyone wants to go back for older versions to check

shyuep commented 5 years ago

Maybe. I am not 100% sure. But we do not have a reliable way to tracking across VASP versions. It would be ideal if VASP themselves issue a list somewhere.

shyuep commented 5 years ago

Also, this will introduce certain fragility in checks. E.g., if I wrote the check based on parameters in the current known VASP parameters, but VASP 6.0 decides to introduce a new one, Custodian will have to decide to (a) remove the parameter or (b) fail. (a) is dangerous since you are modifying a user decision. (b) is no different from VASP just failing.

Actually, I think the better solution is to have a list within pymatgen itself and issue a warning when the INCAR is generated. It does not resolve all issues, but at the least, people are aware what parameters are known and make a decision to ignore the warning.

CifLord commented 5 years ago

I think just having the warnings appear itself would already be very helpful even if it doesn't resolve all the issues as @shyuep said.

I've tested to see what would happen if a nonexistent parameter were inputted into the INCAR and from the vasprun.xml, it looks as though it just copies what was in the INCAR rather than present a list of the actual parameters used in the calculations so that might not help. I've discussed with @ucsdlxg about this and it seems the OUTCAR on the other hand does show that for example if we input METAGGA=Scan for vasp=5.4.1, no METAGGA parameter is present since SCAN doesn't exist for 5.4.1, if on the other hand this was vasp=5.4.4, the OUTCAR shows METAGGA=SCAN. With @mkhorton 's suggestion about enumerating the parameters in the output files, then perhaps we can use the OUTCAR instead of the vasprun.xml as a check:

i.e. if a user INCAR parameter does not show up in the OUTCAR, custodian presents a warning.

It's not as robust as having the actual list from VASP, but it should be able to check for bad parameters?

shyuep commented 5 years ago

Let's do a twin strategy:

  1. Provide a list of current VASP keywords and implement warning check in the Incar class in pymatgen.

  2. Add a check, e.g., OutcarValidator or something, in Custodian which compares INcar and Outcar and says validation failed if the parameter is not present. Validation failure usually is not fatal to using the output of a run. I am not sure whether all valid Incar parameters are always present in the Outcar though.

CifLord commented 5 years ago

This sounds good to me. There's also an comprehensive list of the INCAR parameters on the Vasp manual:

https://cms.mpi.univie.ac.at/wiki/index.php?title=Category:INCAR&pageuntil=ML+FF+LHEAT+MB#mw-pages

which states there are 346 parameters, however I do not known which version of vasp this list would correspond to (I'll asssume the most up to date version). We can use that as a starting point in the first strategy.

mkhorton commented 5 years ago

I wouldn't trust the VASP manual, it's missing some

CifLord commented 5 years ago

I see, will keep in mind, thanks