Support more robust path handling with `remote_workdir`

[Andrew-S-Rosen]

Environment

• Covalent version: 0.234.0rc0
• Covalent-Slurm plugin version: 98e835f29a3bf6b0fa1329051369d461cd81dfe8
• Python version: 3.10+
• Operating system: Ubuntu

What is happening?

Related to #95, when one specifies the remote_workdir with a path like ~/my/dir, it raises an error upon trying to parse the pkl file. The UI shows [Errno 1] : PosixPath('~/test/ef5861af-7b92-4dbc-a52c-eb83bb137496/node_0/result-ef5861af-7b92-4dbc-a52c-eb83bb137496-0.pkl').

How can we reproduce the issue?

import covalent as ct

executor = ct.executor.SlurmExecutor(
    username="rosen",
    address="perlmutter-p1.nersc.gov",
    ssh_key_file="/home/rosen/.ssh/nersc",
    cert_file="/home/rosen/.ssh/nersc-cert.pub",
    conda_env="covalent",
    options={
        "nodes": 1,
        "qos": "debug",
        "constraint": "cpu",
        "account": "matgen",
        "job-name": "test",
        "time": "00:10:00",
    },
    remote_workdir="~/test",
    create_unique_workdir=True,
    cleanup=False,
)

@ct.lattice(executor=executor)
@ct.electron
def workflow():

    return "<3"

ct.dispatch(workflow)()

What should happen?

Filepaths like ~/my/dir should work

Any suggestions?

Wrap the file path in Path().expanduser().resolve() client-side before the electron is launched. As a stretch goal, I would even suggest trying to support something like $SCRATCH/my/dir by wrapping the path call with os.path.expandvars() on the client side too.

You can basically copy what I have done in the covalent-hpc-plugin in terms of file path handling, such as here.