[BUG]: `junifer reset` fails as it tries to delete `junifer_jobs` directory

[fraimondo]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

1. Queue a job (do not submit)
2. Run junifer reset

You see this:

OSError: [Errno 39] Directory not empty: '/home/fraimondo/dev/scratch/neurodc/junifer_jobs'

Expected Behavior

No error, and that directory is not deleted.

Steps To Reproduce

1. Install latest (--pre) from pip
2. Queue any yaml

Environment

junifer:
  version: 0.0.5.dev27
python:
  version: 3.11.8
  implementation: CPython
dependencies:
  click: 8.1.7
  numpy: 1.26.4
  scipy: 1.11.4
  datalad: 0.19.6
  pandas: 2.1.4
  nibabel: 5.2.1
  nilearn: 0.10.2
  sqlalchemy: 2.0.29
  ruamel.yaml: 0.17.40
  httpx: 0.26.0
  tqdm: 4.66.1
  templateflow: 24.2.0
  looseversion: None
system:
  platform: Linux-6.6.13+bpo-amd64-x86_64-with-glibc2.36
environment:
  LC_CTYPE: en_US.UTF-8
  PATH: 
    /home/fraimondo/miniconda3/envs/neurodc/bin:/home/fraimondo/miniconda3/condabin:/home/fraimondo/.dotfiles/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin:/usr/local/games:/usr/games

Relevant log output

2024-04-12 22:35:59,336 - JUNIFER - INFO - ===== Lib Versions =====
2024-04-12 22:35:59,336 - JUNIFER - INFO - numpy: 1.26.4
2024-04-12 22:35:59,336 - JUNIFER - INFO - scipy: 1.11.4
2024-04-12 22:35:59,336 - JUNIFER - INFO - pandas: 2.1.4
2024-04-12 22:35:59,336 - JUNIFER - INFO - nilearn: 0.10.2
2024-04-12 22:35:59,336 - JUNIFER - INFO - nibabel: 5.2.1
2024-04-12 22:35:59,336 - JUNIFER - INFO - junifer: 0.0.5.dev27
2024-04-12 22:35:59,336 - JUNIFER - INFO - ========================
2024-04-12 22:35:59,336 - JUNIFER - INFO - Parsing yaml file: /home/fraimondo/dev/scratch/neurodc/aomic_fc.yaml
2024-04-12 22:35:59,345 - JUNIFER - INFO - Deleting /data/group/riseml/fraimondo/ds003097_FC/ds003097_FC.hdf5
2024-04-12 22:35:59,345 - JUNIFER - INFO - Deleting job directory at /home/fraimondo/dev/scratch/neurodc/junifer_jobs/aomic_fc
Traceback (most recent call last):
  File "/home/fraimondo/miniconda3/envs/neurodc/bin/junifer", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/site-packages/junifer/api/cli.py", line 451, in reset
    api_reset(config)
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/site-packages/junifer/api/functions.py", line 363, in reset
    job_dir.parent.rmdir()
  File "/home/fraimondo/miniconda3/envs/neurodc/lib/python3.11/pathlib.py", line 1156, in rmdir
    os.rmdir(self)
OSError: [Errno 39] Directory not empty: '/home/fraimondo/dev/scratch/neurodc/junifer_jobs'

Anything else?

No response

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (8f96159) to head (b7f6b89).

Additional details and impacted files

[](https://app.codecov.io/gh/juaml/junifer/pull/332?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) ```diff @@ Coverage Diff @@ ## main #332 +/- ## ========================================= Coverage 100.00% 100.00% ========================================= Files 1 1 Lines 1 1 ========================================= Hits 1 1 ``` | [Flag](https://app.codecov.io/gh/juaml/junifer/pull/332/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) | Coverage Δ | | |---|---|---| | [docs](https://app.codecov.io/gh/juaml/junifer/pull/332/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) | `100.00% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml#carryforward-flags-in-the-pull-request-comment) to find out more.

[github-actions[bot]]

PR Preview Action v1.4.7 :---: Preview removed because the pull request was closed. 2024-04-30 11:42 UTC