Advertising the atomate2 paper writing

Zhuoying commented 8 months ago

Dear contributors,

We are thrilled to share the commencement of the Atomate2 paper's preparation phase. Our goal is to have the initial draft ready by our upcoming monthly gathering on March 29.

Since Atomate2 is truly a collective endeavor, we want to ensure that every contributor has the opportunity to contribute to the paper (including those who have made minor contributions but would like to do more to qualify for authorship).

Overview

The paper will be split into 5 sections:

Introduction & Background (@computron)
Atomate2 Design Philosophy & Overview (@utf)
Calculators (team)
Worklows (team)
Conclusion & Future Work (@computron, @utf)

Below, you will find a list of teams categorized according to their workflow and calculator code contributions, compiled to the best of my knowledge. Should there be any oversight or additional workflows you've contributed to that are not mentioned, please do not hesitate to bring them to our attention.

Calculators

VASP (TBD)
CP2K (@nwinner)
ABINIT (@gpetretto, @davidwaroquiers, @VicTrqt, @gmrigna)
FHI-aims (@tpurcell90, @ansobolev)
QCHEM (@rdguha1995, @rkingsbury, @rohithsrinivaas)
Force fields (@matthewkuner @janosh @JaGeo @CompRhys)

We kindly ask each calculator team to prepare the following:

300-600 words of text covering implementation, usage notes, and any existing use case/papers using this implementation.
A summary of the key calculation types implemented.
No figure needed.

Workflows

amset (@utf )
electron phonon (@utf)
lobster (@JaGeo @naik-aakash)
elastic (@mjwen @utf @JaGeo)
harmonic phonon using phonopy (@JaGeo @ab5424)
anharmonic phonon using hiphive (WIP, @hrushikesh-s @Zhuoying)
electrode (@jmmshn)
vasp and cp2k defect (WIP, @jmmshn @nwinner )
aimd and ffmd (WIP, @gpetretto, @mjwen, @davidwaroquiers,@esoteric-ephemera, @chiang-yuan)
openMM (WIP, @orionarcher)
mp (@esoteric-ephemera @Andrew-S-Rosen @janosh)
magnetic orderings (WIP, @mattmcdermott @mkhorton @guymoore13)
adsorption (WIP, @itsduowang)
ferroelectric (WIP, @fraricci)

We kindly ask each workflow team to prepare the following:

500-750 words of text covering methodology, usage notes, and any existing use case/papers using this workflow;
a workflow diagram (will talk more about standardizing the diagram in our next monthly meeting on Mar. 29)
a "pretty" output collage of plots (from raw data calculated from the workflow).

Welcoming Additional Contributors

There are some contributors to atomate2 who have provided fixes here and there but who have not contributed to major workflows. We want to make it clear that these contributors are still welcome to be included on the atomate2 paper. However, we're suggesting that these people could contribute more to the atomate2 code base, either by fixing an open issue or by adding a new tutorial to the documentation. Currently, we could do with the following tutorials, however, if you have other ideas then please let @utf know.

How to write your own atomate2 workflow.
Introduction to the data and document stores.
An introduction to the input sets framework.
An introduction to task documents, schemas, and emmet.
A high level overview of the concepts in atomate2: job/flow makers, input sets, task documents, builders.

If you think you could fit into this category, please reach out to @utf via email and we can chat.

List of tutorial contributors: @QuantumChemist, @ml-evs, @JonathanSchmidt1, @naik-aakash, @JaGeo

Notes

We are delighted to hear your thoughts. For example, if you think anyone should be added to the authorship but not currently listed, don't hesitate to let us know.
You are encouraged to collaborate within or across teams as you see fit. To simplify the collecting process, we propose nominating the first person listed in each workflow as the team leader by default. However, please feel free to adjust the leadership roles within your teams or to merge different workflow teams if you find it necessary. If any changes occur, we would appreciate a brief notification. I will make ongoing updates to the workflow list based on your feedback for a better track.

Point of contact for accessing collaboration platforms

Overleaf: Please leave your email to @utf for access to our Overleaf project. Zoom meeting: Please leave your email to @Zhuoying for the Google invite (our next meeting will be 8:30 AM on March 29 PST to discuss jobflow-remote and the paper writing) Slack channel: Please leave your email to @Andrew-S-Rosen for the Slack invite.

Your contributions are greatly valued. Any of your feedback and thoughts are welcome!

Andrew-S-Rosen commented 8 months ago

I am very excited about this effort and happy to be involved! Apologies for not being able to make the last couple of meetings --- I've had some conflicts.

In any case, @esoteric-ephemera shall I counter-nominate you for the MP workflow lead? 😉 Of course, I am happy to help write.

Zhuoying commented 8 months ago

In any case, @esoteric-ephemera shall I counter-nominate you for the MP workflow lead? 😉 Of course, I am happy to help write.

@Andrew-S-Rosen Thanks for the nomination. I have put @esoteric-ephemera as the first name of the MP workflow.

Hi @mjwen, since @utf and @JaGeo are the lead of a few workflows, are you interested in being the lead of the elastic workflow?

mjwen commented 8 months ago

@Zhuoying Yes, I'd be happy to!

JaGeo commented 8 months ago

Thank you, @Zhuoying . Could you add @ab5424 to the harmonic phonon workflow as well? Thanks! The implemented workflow was based on a previous implementation in his Master thesis.

(To mention shortly, I also only indirectly contributed to the FHI-aims implementation as it starts from the phonon workflow)

utf commented 8 months ago

Thanks very much for putting this message together @Zhuoying. I would just raise two additional points.

Workflows vs Calculators

I think we should make a distinction in the paper between workflows and force/energy calculators. Many of the workflows are agnostic to the choice of calculator (e.g., elastic, phonon, defect etc). I suggest we therefore have a section before workflows providing a short summary for each calculator.

Welcoming additional contributions

There are some contributors to atomate2 who have provided fixes here and there but who have not contributed to major workflows. I want to make it clear that these contributors are still welcome to be included on the atomate2 paper. However, we're suggesting that these people could contribute more to the atomate2 code base, either by fixing an open issue or by adding a new tutorial to the documentation. Currently, I think we could do with the following tutorials, however, if you have other ideas then please let me know.

How to write your own atomate2 workflow.
Introduction to the data and document stores.
An introduction to the input sets framework.
An introduction to task documents, schemas, and emmet (@rkingsbury)
A high level overview of the concepts in atomate2: job/flow makers, input sets, task documents, builders (@QuantumChemist)

If you think you could fit into this category, please reach out to me via email and we can chat.

Notes on the contributor list

Looking through the list, @ansobolev has also contributed to the FHI-aims implementation; @CompRhys added the MACE workflow;

davidwaroquiers commented 8 months ago

Very nice to see this issue open! Just adding @VicTrqt, @gmrigna and @gpetretto to the abinit workflows.

esoteric-ephemera commented 8 months ago

Thanks @Andrew-S-Rosen and @Zhuoying! Might also suggest lumping the EOS workflows (WIP waiting on an emmet release) under the elastic flows.

Would we want to put together an examples directory with jupyter / python notebooks showing typical use? Basically would just mean writing a script to reproduce the figure(s) that we add to the paper

rdguha1995 commented 8 months ago

Thanks @Zhuoying for nominating me and @rkingsbury. This is a question to both @Zhuoying and @utf . For DFT code contributions, do we have a blueprint similar to the one for workflows? Or do we plan to have a generalized code-agnostic discussion in the paper?

tpurcell90 commented 8 months ago

I'm happy to write about the FHI-aims and help out with the phonon workflow parts as well

matthewkuner commented 8 months ago

@JaGeo I wrote up part of the section for forcefields--could you add the relevant parts about phonons/elastic constants to it? We can figure out what kinds of workflow flowcharts we want to do offline (if that works for you)

utf commented 8 months ago

For DFT code contributions, do we have a blueprint similar to the one for workflows?

@rdguha1995 Good question. I think this would be a reasonable starting point:

300-600 words of text covering implementation, usage notes, and any existing use case/papers using this code;
A summary of the key calculation types implemented.
No figure needed;

I'll update the original message with this information.

utf commented 8 months ago

@matthewkuner @JaGeo @janosh what do you think about putting the force field part in the "Calculators" section rather than "Workflows" section. Since the force fields can be used for many workflows I think this makes sense, but happy to hear your thoughts.

janosh commented 8 months ago

Since the force fields can be used for many workflows I think this makes sense

agreed!

JaGeo commented 8 months ago

@matthewkuner @JaGeo @janosh what do you think about putting the force field part in the "Calculators" section rather than "Workflows" section. Since the force fields can be used for many workflows I think this makes sense, but happy to hear your thoughts.

Fine with me!

matthewkuner commented 8 months ago

@matthewkuner @JaGeo @janosh what do you think about putting the force field part in the "Calculators" section rather than "Workflows" section. Since the force fields can be used for many workflows I think this makes sense, but happy to hear your thoughts.

Sounds good to me

JaGeo commented 8 months ago

@utf @Zhuoying Can we maybe use slack channel or a Team to navigate the communication? I feel this will be otherwise hard to organize. (Especially the individual workflow, code teams)

Andrew-S-Rosen commented 8 months ago

@utf @Zhuoying Can we maybe use slack channel or a Team to navigate the communication? I feel this will be otherwise hard to organize. (Especially the individual workflow, code teams)

Seconded. Plus the GitHub notifications are too much for me... 😅

tpurcell90 commented 8 months ago

@utf @Zhuoying Can we maybe use slack channel or a Team to navigate the communication? I feel this will be otherwise hard to organize. (Especially the individual workflow, code teams)

Seconded. Plus the GitHub notifications are too much for me... 😅

I agree with both of these sentiments

Zhuoying commented 8 months ago

@utf @Zhuoying Can we maybe use slack channel or a Team to navigate the communication? I feel this will be otherwise hard to organize. (Especially the individual workflow, code teams)

@JaGeo Sure, it is a good point (always keep a clear organization for efficient communication in such a collaborative project).

@Andrew-S-Rosen, I remember we already have a slack channel called atomate2-working-group created by you a while ago (for monthly meeting purposes). Would you mind using that slack channel (and inviting more contributors to the channel) for the communication?

Andrew-S-Rosen commented 8 months ago

@Andrew-S-Rosen, I remember we already have a slack channel called atomate2-working-group created by you a while ago (for monthly meeting purposes). Would you mind using that slack channel (and inviting more contributors to the channel) for the communication?

I always love when I solved my own problems without remembering 😆 will do!

Andrew-S-Rosen commented 8 months ago

Just added everyone to the Slack (except @ab5424, whose email I need). Feel free to ping @Zhuoying, @utf, or myself to be added if you don't get an email.

JaGeo commented 8 months ago

@Andrew-S-Rosen @ab5424 's mail address is in your mail 😅

tpurcell90 commented 8 months ago

@Andrew-S-Rosen I did not get an email (purcellt@arizona.edu)

QuantumChemist commented 8 months ago

Hey everyone :)

@utf, as we agreed, I will work on an atomate2 tutorial for "A high-level overview of the key concepts in atomate2: job/flow makers, input sets, task documents, and builders" to contribute to this paper :) And could I be added to the slack channel, too, please?

Andrew-S-Rosen commented 8 months ago

@QuantumChemist please send me your email and I'll add you. :)

ml-evs commented 8 months ago

Likewise, discussed with @utf and I will work on some tutorials describing the execution mechanisms (fireworks, JF local, JF remote etc) and would be happy to contribute to the paper. I would also like a slack invite please, hopefully you have my email!

rkingsbury commented 8 months ago

Hi all, thank you for including me in this effort. TBH, I would classify my contributions up to this point as mostly "moral support" to @rdguha1995 plus a few code reviews, so I'm not sure I should qualify as a coauthor (yet). That said, I'd be happy to work with @rdguha1995 on the Q-Chem calculator section and also to develop tutorials on one or both of the following:

An introduction to the input sets framework.
An introduction to task documents, schemas, and emmet.

Plus some tutorials on specific Q-Chem topics (although perhaps not relevant for the paper).

@Andrew-S-Rosen can you please add me to the Slack? I either missed the invitation or did not get one. Is this a dedicated workspace or a channel in BerkeleyTheory or MP?

mkhorton commented 8 months ago

Tagging @guymoore13 for possible inclusion of additional magnetic workflows.

mkhorton commented 8 months ago

@utf, @Zhuoying, @Andrew-S-Rosen please could you invite me via matthew.horton@microsoft.com, using this email since my current/future contributions to atomate2 will be via Microsoft's open source program. Thank you!

Zhuoying commented 8 months ago

Tagging @guymoore13 for possible inclusion of additional magnetic workflows.

I just added @guymoore13 in the main post for the magnetic workflow.

@mkhorton A Google invite was just sent to your Microsoft email. Please feel free to check!

rdguha1995 commented 8 months ago

Tagging @rohithsrinivaas who has been working on developing some flows (primarily frequency flattening optimization) for the qchem calculator.

utf commented 8 months ago

Hi @rkingsbury, that's great to have you on board.

Would you be happy to tackle the tutorial: "An introduction to task documents, schemas, and emmet."?

rkingsbury commented 8 months ago

Hi @rkingsbury, that's great to have you on board.

Would you be happy to tackle the tutorial: "An introduction to task documents, schemas, and emmet."?

Thanks @utf , yes I'd be happy to put that together (dovetails quite well with some ongoing research actually).

As I understand it, the primary "deliverable" here is a contribution to the atomate 2 docs, correct? (as opposed to a Jupyter notebook - although I imagine that could be useful in addition)

rdguha1995 commented 8 months ago

Hi @rkingsbury, that's great to have you on board. Would you be happy to tackle the tutorial: "An introduction to task documents, schemas, and emmet."?

Thanks @utf , yes I'd be happy to put that together (dovetails quite well with some ongoing research actually).

As I understand it, the primary "deliverable" here is a contribution to the atomate 2 docs, correct? (as opposed to a Jupyter notebook - although I imagine that could be useful in addition)

Hey @rkingsbury . Thanks for tackling this! Maybe we should have a chat as well discussing the overall narrative of the QChem calculator section as well.

JonathanSchmidt1 commented 7 months ago

As discussed with @utf I will work on more documentation/tutorial regarding the input sets from an atomate2 perspective and also be happy to contribute to the paper. @Andrew-S-Rosen A slack invite to jonathan.schmidt@mat.ethz.ch would be great.

Andrew-S-Rosen commented 7 months ago

@Zhuoying can you add @JonathanSchmidt1? I won't be at my computer for a few days.

jmmshn commented 7 months ago

@Zhuoying, @utf do we have a tutorial on how to use blob storage? If not I can add that in addition to tutorials for the defect and electrode workflows.

utf commented 7 months ago

Hi @jmmshn, no we don't. Tutorials on those topics would be much appreciated.

jmmshn commented 7 months ago

Also, I don't see a place to put the tutorials in the repo can you point me to the place where notebooks should be kept? Should we just put it under ./docs?

Also, should we require all tutorials to be notebooks and add them to the test suite?
Not sure if that is adding too much work for the contributors but it might not be super bad since the people contributing are likely the ones that wrote the tests in the first place so much of the code can be recycled directly from the tests.

janosh commented 7 months ago

Also, should we require all tutorials to be notebooks and add them to the test suite?

i'm a big fan of testable docs!

JaGeo commented 7 months ago

Oh yes, me too! As everything else breaks with time 🫠

chiang-yuan commented 7 months ago

Upvote for this. If we have templates for testable docs, I will add forcefield md examples to show how to use temperature and pressure scheduling 😃

JaGeo commented 7 months ago

Just also a reminder, there are already existing tutorials by @naik-aakash and me that we can also integrate at some point.

jmmshn commented 7 months ago

So I'm thinking about how to do this. I think this should be doable but I'm not sure what the best way to do this is:

There are two options I see:

Write new mocking code

Since the tests here do not need all the input and output validation (we already do that in the test suite). We can have something like this in the tutorials directory which will be imported by each notebook.

# content of mock.py
from unittest.mock import patch
def mock_vasp(*args):
   # do stuff

patch("<run_vasp_location>", new= mock_vasp).__enter__()
# Similar mocks for other functions as needed.

Pull the current mock functions outside of the current pytest fixtures and just call them as needed we do normal testing.

(1) feels a little more kludgey, especially with the global context setting but it does seem to work with this example:

(2) stay more within the confines of appropriate Python usage, but is a bit more intrusive for both the tutorial writer and reader.

I'm leaning towards (1) but please chime in if anyone feels different. If there are no strong feelings either way I'll find some time next week to get option (1) running.

jmmshn commented 7 months ago

So as it turns out I was wrong and option(2) was much easier for a variety of reasons that were not obvious until I actually tried to do it for 10 mins.

I have implemented option (2) in a basic tutorial for blob storage #776, we can probably extend those trivially to mock the other codes.

tpurcell90 commented 7 months ago

@ansobolev What email should we use to add you to the overleaf paper

ansobolev commented 7 months ago

Hi, sorry for being late, seems that the ping got buried under a bunch of dependabot alerts. Yes, I'd be happy to contribute to FHI-aims implementation of workflows (and also the docs and the paper), @utf can you please add me to the Slack? The e-mail is andrey.n.sobolev@gmail.com.

computron commented 7 months ago

Hi all,

Here is the decision for revised paper structure based on the atomate2 meeting on 3/29/24.

Workflows categorized as "A" would have 2 figures. The first figure details the inputs (both required and optional, like the atomate1 paper) and has a workflow diagram. The second figure details the outputs (an outputs list similar to the inputs, as well as example plots of what you get out of it).

Workflows categorized as "B" will not have figures. However, inputs, workflow description, and outputs will need to part of a text description as well as an overall table.

Paper structure

# Intro
# Design philosophy
# Implementations thus far (calculators and workflows)
# Table of calculators (+1 paragraph per calculator)
# Table of workflows (inputs, outputs) --> these will be detailed in next sections

# Workflows (re-organize by technique (periodic, molecule) --> then A/B within each section)
## Periodic systems
### A
### B
## Molecular systems
### A
### B
## Misc / mixed
### A
### B
# Separate section on MP compatibility 
# Conclusion

A

LOBSTER
Harmonic phonons
FFOpt
Electrode Discovery
Point defect
Classical MD
AIMD (checkpointing)

B

Elastic
Anharmonic phonons
Ferroelectric
Materials Project workflows (workflows themselves separated out, not the MP part)
Magnetic ordering
Adsorption
MPMorph
GW/Excited states

JaGeo commented 7 months ago

Thank you, @computron ! Is there any news on the timeline? This would help us planning the changes (LOBSTER and harmonic phonon part)

computron commented 7 months ago

We want everyone to update their sections by the next meeting (April 26). Overall, I'm expecting it will take until the May meeting to finalize things

@hrushikesh-s can you take care of reorganizing the paper into sections per the discussion? @hrushikesh-s can you also post an example of the type of diagrams expected for workflows of type A? You can just point to a particular workflow section in Overleaf if it helps

materialsproject / atomate2