templateflow / python-client

A python client to query TemplateFlow via pyBIDS
https://templateflow.org/python-client/
Apache License 2.0
8 stars 12 forks source link

FileExistsError from fmriprep #70

Closed jbwexler closed 5 months ago

jbwexler commented 3 years ago

See issue here: https://github.com/nipreps/fmriprep/issues/2505

effigies commented 3 years ago

Bringing the traceback here for reference:

Traceback (most recent call last):
  File "/usr/local/miniconda/bin/fmriprep", line 6, in <module>
    from fmriprep.cli.run import main
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 4, in <module>
    from .. import config
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/config.py", line 96, in <module>
    from templateflow import __version__ as _tf_ver
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/__init__.py", line 19, in <module>
    from . import api
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/api.py", line 7, in <module>
    from .conf import TF_LAYOUT, TF_S3_ROOT, TF_USE_DATALAD
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/conf/__init__.py", line 42, in <module>
    _update_s3(TF_HOME, local=True, overwrite=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/conf/_s3.py", line 20, in update
    retval = _update_skeleton(skel_file, dest, overwrite=overwrite, silent=silent)
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/conf/_s3.py", line 55, in _update_skeleton
    zipref.extractall(str(dest))
  File "/usr/local/miniconda/lib/python3.7/zipfile.py", line 1616, in extractall
    self._extract_member(zipinfo, path, pwd)
  File "/usr/local/miniconda/lib/python3.7/zipfile.py", line 1666, in _extract_member
    os.mkdir(targetpath)
FileExistsError: [Errno 17] File exists: '/scratch1/03201/jbwexler/openneuro_derivatives_tests/derivatives/fmriprep/ds001734-fmriprep/code/containers/binds/HOME/.cache/templateflow/tpl-Fischer344'

I think the fix here is that, when we encounter a FileExistsError in _update_s3(), we should check that the files that were intended to be extracted match checksums and succeed if they do rather than erroring.

Might be a better place for the try/except, though.

oesteban commented 5 months ago

Duplicate of #62. Closing and moving @effigies' idea there.