Resume jobs

[shenwei356]

Trying to resume a finished job. There's a little bug. version: the arglist branch.

$ snakemake --cores all --use-conda --conda-prefix=".conda" -p
Building DAG of jobs...
Nothing to be done (all requested files are present and up to date).
Complete log: .snakemake/log/2024-03-06T132204.711982.snakemake.log
Traceback (most recent call last):
  File "/homes/shenwei/ws/app/miniconda3/envs/miniphy/lib/python3.7/weakref.py", line 648, in _exitfunc
    f()
  File "/homes/shenwei/ws/app/miniconda3/envs/miniphy/lib/python3.7/weakref.py", line 572, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/homes/shenwei/ws/app/miniconda3/envs/miniphy/lib/python3.7/tempfile.py", line 797, in _cleanup
    _shutil.rmtree(name)
  File "/homes/shenwei/ws/app/miniconda3/envs/miniphy/lib/python3.7/shutil.py", line 494, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/homes/shenwei/ws/app/miniconda3/envs/miniphy/lib/python3.7/shutil.py", line 432, in _rmtree_safe_fd
    _rmtree_safe_fd(dirfd, fullname, onerror)
  File "/homes/shenwei/ws/app/miniconda3/envs/miniphy/lib/python3.7/shutil.py", line 432, in _rmtree_safe_fd
    _rmtree_safe_fd(dirfd, fullname, onerror)
  File "/homes/shenwei/ws/app/miniconda3/envs/miniphy/lib/python3.7/shutil.py", line 432, in _rmtree_safe_fd
    _rmtree_safe_fd(dirfd, fullname, onerror)
  [Previous line repeated 5 more times]
  File "/homes/shenwei/ws/app/miniconda3/envs/miniphy/lib/python3.7/shutil.py", line 436, in _rmtree_safe_fd
    onerror(os.rmdir, fullname, sys.exc_info())
  File "/homes/shenwei/ws/app/miniconda3/envs/miniphy/lib/python3.7/shutil.py", line 434, in _rmtree_safe_fd
    os.rmdir(entry.name, dir_fd=topfd)
OSError: [Errno 39] Directory not empty: 'envs

[karel-brinda]

I think this is unrelated to MiniPhy – we use neither of these libraries.

It's again likely some issue related to slow file system synchronization on the cluster.

[shenwei356]

sure.

[karel-brinda]

I think it's probably something related to conda because envs/ is the directory for its environments.