pyiron / pysqa

Simple HPC queuing system adapter for Python on based jinja templates to automate the submission script creation.
https://pysqa.readthedocs.io
BSD 3-Clause "New" or "Revised" License
19 stars 6 forks source link

[feature] Extend the executor support #220

Open jan-janssen opened 11 months ago

jan-janssen commented 11 months ago

Basically migrate the Executor class from pympipool 0.6.X to pysqa.

jan-janssen commented 2 months ago

Add support for resource definition during submission time and function dependence in analogy to pympipool:

jan-janssen commented 2 months ago

Especially in combination with the remote submission using SSH it is important to synchronise the local and the remote cache. To make this cache visible to the user it would be great to create a sub-folder in the current working directory locally. The important part is to also have the ability to link to the output of a previous calculation.

jan-janssen commented 2 months ago

Example calculation for demonstration would be the calculation of a band structure with VASP. This requires two calculation with the WAVECAR being copied from the first to the second calculation. I think the separation of temporary cached data during the execution of the workflow and the storage of the results at the end of the workflow are two very separate tasks.