How to update failed jobs input parameters ?

[naik-aakash]

Hi @davidwaroquiers and @gpetretto ,

Is there any way to query and update failed jobs input parameters in the Jobflow remote?

Example use case, what I mean is as follows

For example, some vasp jobs failed in a flow using atomate2, and I now need to update the jobs' INCAR parameters, for example ALGO/ NPAR, specifically for this failed jobs not complete flow. I cannot just rerun the job; I first need to update these parameters in the DB.

I looked into the code; the only thing I could edit so far was the execution config for already added jobs using JobController. (This is very handy already)

Maybe I am overlooking something. If there exists anything for this, can you please let me know?

[gpetretto]

Hi @naik-aakash, an option that is available at the moment, altough not very good, is the private method JobController._set_job_properties(). This is what is used to set execution config by the method that you mentioned. This should probably be made public. If you think it might be useful I will do this right away. It could also be complemented by an additional method that would directly set values in the job subsection of the document in the DB. Maybe _set_job_properties could be renamed to set_job_doc_properties and add a set_job_values method.

As far as I am concerned, for atomate2 workflows I think it would be useful to have a function like update_job_incar_settings that would directly update the settings of the input set generator for a Vasp Job in the DB. It would be difficult for the generic user to know that one needs to update the job.function.@bound.input_set_generator.user_incar_settings part of the document. I was planning to write such a function myself on the first occasion I would have needed it. Of course such a function cannot sit in jobflow-remote, but I am not sure if it would be fine to have that directly in atomate2 either.

[naik-aakash]

Hi @naik-aakash, an option that is available at the moment, altough not very good, is the private method JobController._set_job_properties(). This is what is used to set execution config by the method that you mentioned. This should probably be made public. If you think it might be useful I will do this right away. It could also be complemented by an additional method that would directly set values in the job subsection of the document in the DB. Maybe _set_job_properties could be renamed to set_job_doc_properties and add a set_job_values method.

As far as I am concerned, for atomate2 workflows I think it would be useful to have a function like update_job_incar_settings that would directly update the settings of the input set generator for a Vasp Job in the DB. It would be difficult for the generic user to know that one needs to update the job.function.@bound.input_set_generator.user_incar_settings part of the document. I was planning to write such a function myself on the first occasion I would have needed it. Of course such a function cannot sit in jobflow-remote, but I am not sure if it would be fine to have that directly in atomate2 either.

Hi @gpetretto , there is already one for vasp in atomate2, but It just returns new job with updated incar settings. Could not figure out how to use it with _set_job_properties . Had came across this method when looking into the codes.

[davidwaroquiers]

Thinking of this, could it be possible to make a replaced job out of a failed one maybe ? Just popped my mind. Of course it's not yet possible but maybe it could be easy to implement ? And (on paper) it seems a good candidate for replacing a job somehow...

[gpetretto]

Hi @gpetretto , there is already one for vasp in atomate2, but It just returns new job with updated incar settings. Could not figure out how to use it with _set_job_properties . Had came across this method when looking into the codes.

I was thinking at an equivalent of the powerups, but geared towards jobflow-remote. Something with an API like:

def update_job_incar_settings(
    db_id: str, 
    incar_updates: dict[str, Any], 
    project_name: str | None = None
):
    ...

that would do the update directly in the DB without the need to go through the JobController explicitely. The implementation could be one of the codes below (preferably option 2?)

To update the INCAR settings there are two options: 1) use the powerup

from jobflow_remote import JobController
from atomate2.vasp.powerups import update_user_incar_settings

jc = JobController.from_project_name("test_project_q")

job_doc = jc.get_job_doc(db_id="75")

job_doc.job = update_user_incar_settings(job_doc.job, {"NPAR": 4})

jc._set_job_properties(job_doc.as_db_dict(), db_id="75")

# check that it has been updated correctly
job_doc_mod = jc.get_job_doc(db_id="75")
print(job_doc_mod.job.maker.input_set_generator.user_incar_settings)

2) change only the target attribute:

from jobflow_remote import JobController

jc = JobController.from_project_name("test_project_q")

jc._set_job_properties({"job.function.@bound.input_set_generator.user_incar_settings.NPAR": 2}, db_id="75")

job_doc_mod = jc.get_job_doc(db_id="75")
print(job_doc_mod.job.maker.input_set_generator.user_incar_settings)

would one of these work for you?

[gpetretto]

Thinking of this, could it be possible to make a replaced job out of a failed one maybe ? Just popped my mind. Of course it's not yet possible but maybe it could be easy to implement ? And (on paper) it seems a good candidate for replacing a job somehow...

I am not sure I understand exactly this point. Do you mean that jobflow-remote should create a replacement job, even if jobflow does not? In that case I think it could lead to problems of conficts between jobflow and jobflow-remote that could be difficult to foresee and test. Also, changing the incar settings and rerun the job does not seem particularly more difficult than generating a replacement with a specific command and change the incar settings there. Actually the latter seems a more involved procedure. I would rather push for a better handling directly in atomate2 (e.g. intercepting of the walltime and setting up a continuation job).

[naik-aakash]

Yeah, I think the option with I was thinking at an equivalent of the powerups, but geared towards jobflow-remote. Something with an API like:

Hi @gpetretto , there is already one for vasp in atomate2, but It just returns new job with updated incar settings. Could not figure out how to use it with _set_job_properties . Had came across this method when looking into the codes.

I was thinking at an equivalent of the powerups, but geared towards jobflow-remote. Something with an API like:

def update_job_incar_settings(
    db_id: str, 
    incar_updates: dict[str, Any], 
    project_name: str | None = None
):
    ...

that would do the update directly in the DB without the need to go through the JobController explicitely. The implementation could be one of the codes below (preferably option 2?)

To update the INCAR settings there are two options:

1. use the powerup

from jobflow_remote import JobController
from atomate2.vasp.powerups import update_user_incar_settings

jc = JobController.from_project_name("test_project_q")

job_doc = jc.get_job_doc(db_id="75")

job_doc.job = update_user_incar_settings(job_doc.job, {"NPAR": 4})

jc._set_job_properties(job_doc.as_db_dict(), db_id="75")

# check that it has been updated correctly
job_doc_mod = jc.get_job_doc(db_id="75")
print(job_doc_mod.job.maker.input_set_generator.user_incar_settings)

2. change only the target attribute:

from jobflow_remote import JobController

jc = JobController.from_project_name("test_project_q")

jc._set_job_properties({"job.function.@bound.input_set_generator.user_incar_settings.NPAR": 2}, db_id="75")

job_doc_mod = jc.get_job_doc(db_id="75")
print(job_doc_mod.job.maker.input_set_generator.user_incar_settings)

would one of these work for you?

Yes, I think powerups equivalent to atomate2 in job flow remote would be very convenient to have, as you suggested. Both the options seem fine to me. Am a bit more inclined to option 1, though as it seems bit less involved not having to know the complete nested dict structure. Thanks again for the suggestions 😄

[gpetretto]

Good 😄
Since it seems useful I will make _set_job_properties public, leaving the original there with a deprecation warning.

In general, given that such a function would be bound to atomate2 and VASP I don't think that jobflow-remote would be a good place for it. @utf do you think it would make sense to have this directly in atomate2? Or would it be better to create a new atomate2-jfremote package for this purpose?