Atoms plugin in Larch?

[maurov]

Does anybody is planning to port "atoms" to Larch? If not, I could probably invest some time in this. I never went through the source code of atoms itself, so the ideas I have in mind on how to implement it could result not compatible with Demeter then...

The Larch plugin I have in mind should simply do: 1) read a CIF file and some keyword parameters; 2) based on this, it will return a FEFF* or FDMNES input file.

For the templates, I think is enough the 'string.Template' included in the Python standard library (there are tons of powerful template systems available, but this will avoid to include a further dependence in Larch).

For the CIF parsing, well, I think we are obliged to include a dependence on an external library. Do you have any preference for this? I'm evaluating some of them and at the moment the one I like the most is in ASE (https://wiki.fysik.dtu.dk/ase/index.html)... well, this is using a sledgehammer to crack a nut.

Anyway, your feedback is welcome.

[newville]

Hi Mauro,

On Sat, Jun 15, 2013 at 8:14 AM, Mauro Rovezzi notifications@github.com wrote:

Does anybody is planning to port "atoms" to Larch? If not, I could probably invest some time in this. I never went through the source code of atoms itself, so the ideas I have in mind on how to implement it could result not compatible with Demeter then...

This wasn't really on my To Do list, but I think it would be great to have some way to generate and manipulate Feff.inp etc, especially if this can tie into other tools.

I'm slightly reluctant to step on Bruce's work, as he's put a ton of effort and support in to Atoms, Demeter, etc. Still, having this kind of functionality directly in Python and Larch has appeal, especially if it allows tying to other Python packages, and there are so many scientific python packages, we have to use them...

The Larch plugin I have in mind should simply do: 1) read a CIF file and some keyword parameters; 2) based on this, it will return a FEFF* or FDMNES input file.

That would be great. FWIW, we're hoping / expecting to get a newer version of Feff-for-EXAFS released under a BSD-ish license from John Rehr and Josh Kas sometime soon (I spoke with Josh about this recently, and he agreed to get on it). At that point, ripping apart the input file and i/o structure, including replacing the Pathfinder will be on my To Do list, and doing that with Python/Larch would be fantastic, especially if it allows tying to other packages (molecular viewer?).

For the templates, I think is enough the 'string.Template' included in the Python standard library (there are tons of powerful template systems available, but this will avoid to include a further dependence in Larch).

I agree with all of that. Adding dependencies to Larch shouldn't be done too lightly, but isn't necessarily a problem. Having a plugin that needs an external dependency is definitely OK.

For the CIF parsing, well, I think we are obliged to include a dependence on an external library. Do you have any preference for this? I'm evaluating some of them and at the moment the one I like the most is in ASE (https://wiki.fysik.dtu.dk/ase/index.html)... well, this is using a sledgehammer to crack a nut.

I know of a Python project for CIF files from James Hester at https://bitbucket.org/jamesrhester/pycifrw/ . I've not tried to use this for any real work, especially involving turning Crystallographic data into atomic coordinates. That might give it a smaller dependency, but the PyCIFRW project seems to be more about validating complex CIF files than actually working with crystallographic data.

I've not done anything with ASE, but looks very interesting, and like it's probably just the right tool for the job. Adding it as a dependency might be useful for other needs too.

--Matt

[bruceravel]

There is no danger of stepping on my toes. If larch grows and becomes a real thing and someone were to take a stab at replacing Athena and Artemis, I would consider that a good thing. I've been supporting an oddly large portion of the XAS community for 12 years now. (Certainly not alone, but in a way that is different from what Matt and others have done.) While that has been a very good thing for my career and while I still have energy to continue doing so for now, the day is inevitably going to come when I just won't have it in me to carry on. If larch can be an ecosystem in which the successor to A&A can evolve -- wonderful!

[bruceravel]

If you are willing to consider dependencies, I would strongly recommend http://openbabel.org/ be used in some capacity. The huge advantage of doing so is that your effort would not be limited to Feff and FDMNES, although it would require writing OpenBabel plugins to support Feff and FDMNES. It would also open the possibility of piping things like PD files directly into a Feff calculation.

In fact, parceling the chore of reading CIF out to OpenBabel and having it write the Feff/FDMNES input file would obviate the difficult chore of dealing with CIF. It ships with good python bindings.

[newville]

On Jun 15, 2013 1:12 PM, "Bruce Ravel" notifications@github.com wrote:

If you are willing to consider dependencies, I would strongly recommend http://openbabel.org/ be used in some capacity. The huge advantage of doing so is that your effort would not be limited to Feff and FDMNES, although it would require writing OpenBabel plugins to support Feff and FDMNES. It would also open the possibility of piping things like PD files directly into a Feff calculation.

That's a really good suggestion. Being able to hook to other formats and viewers is a huge advantage of OpenBabel. I think the package Mauro pointed to can also do some CIF parsing, and then ship off atomic coords to some of the common DFT and MD simulations programs. Hooking to a molecular viewer (jmol?) would be nice....

In fact, parceling the chore of reading CIF out to OpenBabel and having it write the Feff/FDMNES input file would obviate the difficult chore of dealing with CIF. It ships with good python bindings.

I don't know how the package Mauro suggested compares with OpenBabel for parsing CIF, but it claims to be able to do it.....

Anyway, having some support for CIF -> atomic coords for input file of program X would be highly useful.

--Matt

[maurov]

Matt, Bruce,

Thank you very much indeed for your useful comments. First of all, let me say that Bruce is doing a wonderful work with his codes and certainly I do not plan to step on them. In fact, I do not have the time to code all the ideas I have and I'm an happy user of Atoms and friends. Currently, I use its command line version to convert CIFs to FEFF via a given sets of Atoms templates. This workflow is nice, but still I have sometime to modify manually either the Atoms templates, or the FEFF inputs. So, bringing a CIF parser in Larch seems to me a good feature to fully automatize the process of moving from XRD and DFT to spectroscopy.

Here some comments about the CIF libraries:

• Openbabel is in my evaluation list. I know how powerful is this project but at the same time it requires a big effort for entering their ecosystem (especially for someone like me who thinks in terms of crystals, not molecules). Unfortunately, at my last trial I got stuck between Pybel and Openbabel.py, so I have to read again the Python bindings documentation (http://openbabel.org/docs/current/UseTheLibrary/Python.html)
• ASE has some advantages over the basic features we are searching for:
  • is a Python project, everything is directly accessible, not need for a limited API
  • it is shipped with a simple Effective Medium Potential (EMT) for quick minimizations of the input structures or one can directly plug-in strong calculators (particularly useful for combining molecular dynamics and EXAFS)
  • the CIF or the generated list of atoms can be directly visualized and modified with its own GUI
  • supercells and periodic systems are supported very nicely
• CCTBX also has a nice CIF parser for Python (http://cctbx.sourceforge.net/iotbx_cif/). I like CCTBX for diffraction, but I'm not going in that direction at the moment ;)
• PyCIFRW... not evaluated yet

OK, I let you know more if I come up with something (it is possible that I will not manage to find the time to code all this)...

[maurov]

I just found an interesting project that seems to build on top of Openbabel, ASE and PyCIFRW: pymatgen

[newville]

Hi Mauro,

On Sun, Jun 16, 2013 at 5:35 AM, Mauro Rovezzi notifications@github.com wrote:

I just found an interesting project that seems to build on top of Openbabel, ASE and PyCIFRW: pymatgen

— Reply to this email directly or view it on GitHub.

Yes, that looks very interesting -- and a very active project that can tie into lots of theoretical codes. Wow, they seem to already have support for Feff:

https://github.com/materialsproject/pymatgen/blob/master/pymatgen/io/feffio.py https://github.com/materialsproject/pymatgen/blob/master/scripts/feff_input_generation.py

Definitely worth knowing about and figuring out how to tie into, depend on, etc. --Matt

[maurov]

@newville I think this is exactly what cif2feff does, closing then!

[newville]

@maurov yeah, it took a long time and it ended up using pymatgen as you suggested years ago - I did end up borrowing from but sort of rewriting the pymatgen 'convert CIF structure to feff input'. It can write feff8 or feff6 input, and tries to handle fractional occupancy by weighted random selection.

Mostly, though, I wanted to replace the need for webatoms and include a collection of CIF files. There is now an Sqlite3 database "larch/xrd/amscif_trim.db" (~10Mb) that has ~9K CIF structures from the American Mineralogical collection. If the user is connected to the internet and can access https://millenia.cars.aps.anl.gov, there is a "full" version of this collection, with ~20K structures files (~30Mb) that will be downloaded (or try to, falling back to the "trimmed" version. FWIW, the "trimming" removes some obvious repeats, as from P- or T-dependent structures in a single publication and structures with more than 12 atoms/sites.

Whichever version is used, this collection can be searched with a "cif browser" GUI form (available from XAS Viewer's Feff menu). With that, CIF files can be searched and selected and can be converted to Feff input files (which can also be edited in place, of course), and then Feff8 or Feff6 run, with runs in separate folders under $HOME/.larch/feff/. The CIFs can also display a 1D XRD pattern, though I don't know how useful that would be. I believe there isn't a good python/OpenGL CIF renderer.... It can also import CIF files, but that seems very scary to me, and CIFs are extremely fragile.

Anyway, this CIF browser / Feff Runner seems "good enough for initial release" to me, but if you have a chance to try it out and make suggestions, that would be most welcome!

Still a work-in-progress, and there is not a "Feffit" tab on XAS Viewer yet, but it might be soon.

[maurov]

@newville for what I have tested, the "cif browser" and FEFF runner work just fine. I suggest only one (simple) extra feature: #301.

Regarding the CIF 3D rendering/visualization, I do not think it is a good idea to invest efforts on it, as there are many software out there that work just greatly and the user can use the one of choice to generate the CIF file. Rendering the 3D structure would make sense only to visualize the generated FEFF paths, but I am afraid this is not an easy task. We could build on top of pymatgen.vis. VTK could be integrated in Wx, but it seems there is more momentum toward WebGL.

[newville]

@maurov for structure visualization: yes that can come later, possibly much, much later ;)