datalad / datalad-ukbiobank

Resources for working with UKBiobank as a DataLad dataset
MIT License
6 stars 12 forks source link

git-annex 8.20210223+git33-gee4fd38ec breaks things #65

Closed mih closed 3 years ago

mih commented 3 years ago
======================================================================
ERROR: datalad_ukbiobank.tests.test_update.test_drop
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/datalad_ukbiobank/tests/test_update.py", line 195, in test_drop
    ds.ukb_update(merge=True, force=True, drop='archives')
Obscure filename: str=b' |;&%b5{}\'"\xce\x94\xd0\x99\xd7\xa7\xd9\x85\xe0\xb9\x97\xe3\x81\x82 .datc ' repr=' |;&%b5{}\'"ΔЙקم๗あ .datc '
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/datalad/distribution/dataset.py", line 503, in apply_func
Encodings: default='utf-8' filesystem='utf-8' locale.prefered='UTF-8'
    return f(**kwargs)
Environment: LANG='C.UTF-8' PATH='/opt/hostedtoolcache/Python/3.7.10/x64/bin:/opt/hostedtoolcache/Python/3.7.10/x64:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/opt/pipx_bin:/usr/share/rust/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/snap/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin' GIT_CONFIG_PARAMETERS="'init.defaultBranch=master'"
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/datalad/interface/utils.py", line 482, in eval_func
    return return_func(generator_func)(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/datalad/interface/utils.py", line 470, in return_func
    results = list(results)
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/datalad/interface/utils.py", line 401, in generator_func
    allkwargs):
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/datalad/interface/utils.py", line 557, in _process_results
    for res in results:
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/datalad_ukbiobank/update.py", line 289, in __call__
    for rec in repo.call_annex_records(['drop'] + drop_opts):
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/datalad/support/annexrepo.py", line 1206, in call_annex_records
    return self._call_annex_records(args, files=files)
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/datalad/support/annexrepo.py", line 1127, in _call_annex_records
    raise e
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/datalad/support/annexrepo.py", line 1087, in _call_annex_records
    **kwargs,
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/datalad/support/annexrepo.py", line 987, in _call_annex
    **kwargs)
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/datalad/cmd.py", line 413, in run
    **results,
datalad.support.exceptions.CommandError: CommandError: 'git annex drop --force --branch incoming -I '*.zip' --json --json-error-messages -c annex.dotfiles=true -c annex.retry=3' failed with exitcode 1 under /tmp/datalad_temp_test_drop6c80q7z0 [err: 'git-annex: Cannot use --all or --unused or --key or --incomplete with options that match on file names.'] [info keys: stdout_json]
bpoldrack commented 3 years ago

Hm. What is the -I option? Seems undocumented (I'm running 8.20201127). May be it's meaning changed (since it was not documented to begin with)?

mih commented 3 years ago

from git annex drop -h

  -I,--include GLOB        limit to files matching the glob pattern
bpoldrack commented 3 years ago

Interesting. Short option doesn't show up in the man pages (git-annex-matching-(options|expression)).

mih commented 3 years ago

Tried with a newer 8.20210310 and it does not (or no longer) replicate.