problematic parsing of *.traj files from ase

[schumannj]

Something seems to have been changed between 2022 and February 2023. Earlier *.traj seemed to have been parsed correctly, i.e.at least the structure was extracted. See e.g. here, for the same file the structure is not extracted currently. Since last year, even though also the asap parser recognised the file, the structure was not extracted, see .e.g. here

[JFRudzinski]

@ladinesa @ndaelman-hu I talked to Julia about this, and it appears now that the ".traj" parser (i.e., asap parser) has been somehow broken and then fixed in the mean time.

In our discussion though I began to wonder the scope of this parser. My understanding is that the ".traj" format is ase's native trajectory format. Is there other explicit support for ase?

In these examples that Julia posted, I believe that ase is executing FHIAims or VASP. I just wonder what was the original functionality intended when implementing the asap parser, and if we should consider extending this (probably largely depends on how relevant this is for you @schumannj). Julia and I talked about the user creating a workflow file when running such calculations with ase.

[ladinesa]

as far as I can remember, only ASAP parses traj files, maybe we could restrict more the file contents so as to exclude unintended files.

[ndaelman-hu]

it appears now that the ".traj" parser (i.e., asap parser) has been somehow broken and then fixed in the mean time.

Yeah, I identified this problem a while ago and passed it on to @Bernadette-Mohr. I think she solved it already.

My understanding is that the ".traj" format is ase's native trajectory format. Is there other explicit support for ase?

ase is really more a workflow parser, not an atomistic one. Its main purpose is to interface with (originally DFT) simulation packages. What distingusihes it from other workflow managers is that also allows the algorithmic creation and manipulation of the model system.

In these examples that Julia posted, I believe that ase is executing FHIAims or VASP.

The old error log traces back to lines 44-46 in the parser: it's looking for a calculator object to verify the integrity of the trajectory. This is what was failing with Julia's calculations. My impression is that this requirement is too strong, but maybe @Bernadette-Mohr can share her opinion.

[JFRudzinski]

ok, thank you everyone for the clarification, I think I understand now.

@schumannj I think it would be good to take a look at any use-cases that you have and try to understand better the typical use/appearance of these .traj files. It may be that we can extend the ASAP parser in some way to recognize some standard workflows (depending on the available information). Alternatively, we can approach it by creating workflow files to describe the connection between these entries, as you and I discussed previously.

[schumannj]

@ladinesa I have added a couple of different examples for .traj files to the branch and created folders for a seperate ase parser. 2 of the traj files I took from published NOMAD uploads, so we have examples of traj files with different calculators and level of information.

[ladinesa]

so i looked at the issue and here's my proposal:

1. create a common ase traj parser in simulation-parsers
2. create a new ase-workflow? parser in workflow-parsers
3. use ase traj parser for the current asap and ase-workflow parsers.

I can set up the skeleton for these and perhaps parse some basic quantities.

[JFRudzinski]

Sounds good to me. @schumannj after Alvin implements the basic support, we could come back together and assist you with specific workflows and such I think :grin:

[schumannj]

Sounds good to me as well. Yes, I will definitely need some help for the workflows. Thanks!

[ladinesa]

change of plan, i would put everything in atomistic-parsers, so as not to introduce the complication of spreading codes in 3 plugins.

[ladinesa]

there seems to be a technical issue with the entry point not taking in mainfile_binary_header(_re) so until it is not fixed, asap parser will still match all traj files.

[ladinesa]

The issue has been fixed, ase traj files are now parsed as ase. @schumannj can you please test this with nomad-develop?

[schumannj]

@ladinesa I can test this. You mean locally, right? how do I add the branch of the atomisitc parser? Do I have to replace nomad-parser-plugins-atomistic with the local atomistic-parsers using pip uninstall/install from local?

[ladinesa]

creating

yes pls, just do a uv pip install atomistic-parsers, after running the nomad setup so you use your local atomisparsers. maybe you can also try to extend the ase parser now since I only implemented the parsing of common quantities.

[schumannj]

it works now for traj files which are a direct result of a vasp calculation, however for some other examples I get an error:

[ladinesa]

it works now for traj files which are a direct result of a vasp calculation, however for some other examples I get an error: !

just pushed a fix

[schumannj]

And for the files it parses correctly, the calculator 'vasp' is put under force_fields. I am not sure how to solve this, as ase could be used for both force_fields and dft calculations. Maybe the name could be checked against existing dft codes?

This is where the workflow parsing goes in. I can work on this if a specific dataset example is given to me otherwise I would refer this to @JFRudzinski

[schumannj]

it works now for traj files which are a direct result of a vasp calculation, however for some other examples I get an error: !

just pushed a fix

I can confirm, now it works.