search

psychoinformatics-de / datalad-hirni

DataLad extension for (semi-)automated, reproducible processing of (medical/neuro)imaging data
http://datalad.org
Other
5 stars 8 forks source link

AnnexBatchCommandError when performing hirni-import-dcm #201

Open m-petersen opened 3 years ago

m-petersen commented 3 years ago

Dear Developers,

currently I am trying to implement Datalad Hirni for raw dataset creation and BIDS conversion of some test data. Performing hirni-import-dcm on some DCMs (I have previously converted to tarballs with "tar cvfz DCM.tar.gz DCM/") yields the following error I don't know how to address:

AnnexBatchCommandError: 'addurl' [Error, annex reported failure for addurl (url='dl+archive:MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz#path=T1_MPRAGE_COR_ND_0008/DCM_102.MR.STUDIEN_NEUROLOGIE.0008.0188.2018.12.18.12.07.27.214150.163607951.IMA&size=287232'): {'command': 'addurl', 'note': 'from datalad-archives\nto T1_MPRAGE_COR_ND_0008/DCM_102.MR.STUDIEN_NEUROLOGIE.0008.0188.2018.12.18.12.07.27.214150.163607951.IMA', 'success': False, 'input': ['dl+archive:MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz#path=T1_MPRAGE_COR_ND_0008/DCM_102.MR.STUDIEN_NEUROLOGIE.0008.0188.2018.12.18.12.07.27.214150.163607951.IMA&size=287232 T1_MPRAGE_COR_ND_0008/DCM_102.MR.STUDIEN_NEUROLOGIE.0008.0188.2018.12.18.12.07.27.214150.163607951.IMA'], 'error-messages': [" Failed to fetch any archive containing URL-s287232--dl,43archive:MD5E-s72842081--34-71d3f8069a40ce1960d8450c16d73b15. Tried: ['MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz', 'MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz', 'MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz'] [archives.py:_transfer:409]"], 'file': 'T1_MPRAGE_COR_ND_0008/DCM_102.MR.STUDIEN_NEUROLOGIE.0008.0188.2018.12.18.12.07.27.214150.163607951.IMA'}]

Command: datalad -l 8 hirni-import-dcm --anon-subject 001 $sub-test/ses-1/T1_MPRAGE_COR_ND_0008.tar.gz 001_struct

Happy for any input and to provide further information.

Sincerely, Marvin

Full output using the -l 8 flag is:

[DEBUG ] Command line args 1st pass for DataLad 0.14.4. Parsed: Namespace() Unparsed: ['hirni-import-dcm', '--anon-subject', '001', '../../sub-test/ses-1/T1_MPRAGE_COR_ND_0008.tar.gz', '001_struct'] [DEBUG ] Discovering plugins [DEBUG ] Loading entrypoints [DEBUG ] Loading entrypoint webapp from datalad.extensions for docs building [DEBUG ] Building doc for <class 'datalad_webapp.WebApp'> [DEBUG ] Loaded entrypoint webapp [DEBUG ] Loading entrypoint ukbiobank from datalad.extensions for docs building [DEBUG ] Loaded entrypoint ukbiobank [DEBUG ] Loading entrypoint neuroimaging from datalad.extensions for docs building [DEBUG ] Loaded entrypoint neuroimaging [DEBUG ] Loading entrypoint metalad from datalad.extensions for docs building [DEBUG ] Loaded entrypoint metalad [DEBUG ] Loading entrypoint hirni from datalad.extensions for docs building [DEBUG ] Loaded entrypoint hirni [DEBUG ] Loading entrypoint container from datalad.extensions for docs building [DEBUG ] Loaded entrypoint container [DEBUG ] Building doc for <class 'datalad.core.local.status.Status'> [DEBUG ] Building doc for <class 'datalad.core.local.save.Save'> [DEBUG ] Building doc for <class 'datalad.interface.annotate_paths.AnnotatePaths'> [DEBUG ] Building doc for <class 'datalad_metalad.dump.Dump'> [DEBUG ] Building doc for <class 'datalad_hirni.commands.spec4anything.Spec4Anything'> [DEBUG ] Building doc for <class 'datalad_hirni.commands.dicom2spec.Dicom2Spec'> [DEBUG ] Building doc for <class 'datalad_metalad.extract.Extract'> [DEBUG ] Building doc for <class 'datalad.core.local.diff.Diff'> [DEBUG ] Building doc for <class 'datalad_metalad.aggregate.Aggregate'> [DEBUG ] Building doc for <class 'datalad.interface.download_url.DownloadURL'> [DEBUG ] Building doc for <class 'datalad_hirni.commands.import_dicoms.ImportDicoms'> [DEBUG ] Parsing known args among ['/work/fatx405/miniconda3/bin/datalad', '-l', '8', 'hirni-import-dcm', '--anon-subject', '001', '../../sub-test/ses-1/T1_MPRAGE_COR_ND_0008.tar.gz', '001_struct'] [DEBUG ] Async run: | cwd=None | cmd=['git', '--git-dir=', 'config', '-z', '-l', '--show-origin'] [DEBUG ] Launching process ['git', '--git-dir=', 'config', '-z', '-l', '--show-origin'] [DEBUG ] Process 4538 started [DEBUG ] Waiting for process 4538 to complete [DEBUG ] Process 4538 exited with return code 0 [Level 8] Finished running ['git', '--git-dir=', 'config', '-z', '-l', '--show-origin'] with status 0 [DEBUG ] Determined class of decorated function: <class 'datalad_hirni.commands.import_dicoms.ImportDicoms'> [DEBUG ] Resolved dataset for import DICOM session: /work/fatx405/projects/2106_test_proc/sourcedata [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata | cmd=['git', 'config', '-z', '-l', '--show-origin'] [DEBUG ] Launching process ['git', 'config', '-z', '-l', '--show-origin'] [DEBUG ] Process 4578 started [DEBUG ] Waiting for process 4578 to complete [DEBUG ] Process 4578 exited with return code 0 [Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata | cmd=['git', 'config', '-z', '-l', '--show-origin', '--file', '/work/fatx405/projects/2106_test_proc/sourcedata/.datalad/config'] [DEBUG ] Launching process ['git', 'config', '-z', '-l', '--show-origin', '--file', '/work/fatx405/projects/2106_test_proc/sourcedata/.datalad/config'] [DEBUG ] Process 4611 started [DEBUG ] Waiting for process 4611 to complete [DEBUG ] Process 4611 exited with return code 0 [Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin', '--file', '/work/fatx405/projects/2106_test_proc/sourcedata/.datalad/config'] with status 0 [DEBUG ] Discovering plugins [DEBUG ] Building doc for <class 'datalad.core.local.create.Create'> [DEBUG ] Found matching interface ('datalad.core.local.create', 'Create') for create [DEBUG ] Determined class of decorated function: <class 'datalad.core.local.create.Create'> [DEBUG ] Async run: | cwd=None | cmd=['git', '--git-dir=', 'config', '-z', '-l', '--show-origin'] [DEBUG ] Launching process ['git', '--git-dir=', 'config', '-z', '-l', '--show-origin'] [DEBUG ] Process 4662 started [DEBUG ] Waiting for process 4662 to complete [DEBUG ] Process 4662 exited with return code 0 [Level 8] Finished running ['git', '--git-dir=', 'config', '-z', '-l', '--show-origin'] with status 0 [DEBUG ] Query status of GitRepo('/work/fatx405/projects/2106_test_proc/sourcedata') for 1 paths [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] [DEBUG ] Process 4694 started [DEBUG ] Waiting for process 4694 to complete [DEBUG ] Process 4694 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] with status 0 [DEBUG ] GitRepo(/work/fatx405/projects/2106_test_proc/sourcedata).get_content_info(...) [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'config', '-z', '-l', '--file', '.gitmodules'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'config', '-z', '-l', '--file', '.gitmodules'] [DEBUG ] Process 4743 started [DEBUG ] Waiting for process 4743 to complete [DEBUG ] Process 4743 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'config', '-z', '-l', '--file', '.gitmodules'] with status 0 [DEBUG ] GitRepo(/work/fatx405/projects/2106_test_proc/sourcedata).get_content_info(...) [DEBUG ] Query repo: ['ls-files', '--stage', '-z'] [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z'] [DEBUG ] Process 4798 started [DEBUG ] Waiting for process 4798 to complete [DEBUG ] Process 4798 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z'] with status 0 [DEBUG ] Done query repo: ['ls-files', '--stage', '-z'] [DEBUG ] Done GitRepo(/work/fatx405/projects/2106_test_proc/sourcedata).get_content_info(...) [DEBUG ] Query repo: ['ls-files', '--stage', '-z'] [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z', '--', '001_struct/dicoms'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z', '--', '001_struct/dicoms'] [DEBUG ] Process 4835 started [DEBUG ] Waiting for process 4835 to complete [DEBUG ] Process 4835 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z', '--', '001_struct/dicoms'] with status 0 [DEBUG ] Done query repo: ['ls-files', '--stage', '-z'] [DEBUG ] Done GitRepo(/work/fatx405/projects/2106_test_proc/sourcedata).get_content_info(...) [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '-z', '-m', '--', '001_struct/dicoms'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '-z', '-m', '--', '001_struct/dicoms'] [DEBUG ] Process 4869 started [DEBUG ] Waiting for process 4869 to complete [DEBUG ] Process 4869 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '-z', '-m', '--', '001_struct/dicoms'] with status 0 [DEBUG ] GitRepo(/work/fatx405/projects/2106_test_proc/sourcedata).get_content_info(...) [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'config', '-z', '-l', '--file', '.gitmodules'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'config', '-z', '-l', '--file', '.gitmodules'] [DEBUG ] Process 4901 started [DEBUG ] Waiting for process 4901 to complete [DEBUG ] Process 4901 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'config', '-z', '-l', '--file', '.gitmodules'] with status 0 [DEBUG ] GitRepo(/work/fatx405/projects/2106_test_proc/sourcedata).get_content_info(...) [DEBUG ] Query repo: ['ls-files', '--stage', '-z'] [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z'] [DEBUG ] Process 4938 started [DEBUG ] Waiting for process 4938 to complete [DEBUG ] Process 4938 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z'] with status 0 [DEBUG ] Done query repo: ['ls-files', '--stage', '-z'] [DEBUG ] Done GitRepo(/work/fatx405/projects/2106_test_proc/sourcedata).get_content_info(...) [DEBUG ] Query repo: ['ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l'] [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l', '--', '001_struct/dicoms'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l', '--', '001_struct/dicoms'] [DEBUG ] Process 4973 started [DEBUG ] Waiting for process 4973 to complete [DEBUG ] Process 4973 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l', '--', '001_struct/dicoms'] with status 0 [DEBUG ] Done query repo: ['ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l'] [DEBUG ] Done GitRepo(/work/fatx405/projects/2106_test_proc/sourcedata).get_content_info(...) [INFO ] Creating a new annex repo at /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms [DEBUG ] Initialize empty Git repository at '/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms' [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', '-C', '/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms', 'init'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', '-C', '/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms', 'init'] [DEBUG ] Process 5009 started [DEBUG ] Waiting for process 5009 to complete [DEBUG ] Process 5009 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', '-C', '/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms', 'init'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'for-each-ref', '--format=%(refname:strip=2)', 'refs/heads', 'refs/remotes'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'for-each-ref', '--format=%(refname:strip=2)', 'refs/heads', 'refs/remotes'] [DEBUG ] Process 5052 started [DEBUG ] Waiting for process 5052 to complete [DEBUG ] Process 5052 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'for-each-ref', '--format=%(refname:strip=2)', 'refs/heads', 'refs/remotes'] with status 0 [DEBUG ] Initializing annex repository at /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms... [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', 'config', '-z', '-l', '--show-origin'] [DEBUG ] Launching process ['git', 'config', '-z', '-l', '--show-origin'] [DEBUG ] Process 5087 started [DEBUG ] Waiting for process 5087 to complete [DEBUG ] Process 5087 exited with return code 0 [Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'for-each-ref', '--format=%(refname:strip=2)', 'refs/heads'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'for-each-ref', '--format=%(refname:strip=2)', 'refs/heads'] [DEBUG ] Process 5120 started [DEBUG ] Waiting for process 5120 to complete [DEBUG ] Process 5120 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'for-each-ref', '--format=%(refname:strip=2)', 'refs/heads'] with status 0 [DEBUG ] Launching process ['/work/fatx405/miniconda3/bin/python', '--version'] [DEBUG ] Process 5170 started [DEBUG ] Waiting for process 5170 to complete [DEBUG ] Process 5170 exited with return code 0 [DEBUG ] Async run: | cwd=None | cmd=['git', 'annex', 'version', '--raw'] [DEBUG ] Launching process ['git', 'annex', 'version', '--raw'] [DEBUG ] Process 5172 started [DEBUG ] Waiting for process 5172 to complete [DEBUG ] Process 5172 exited with return code 0 [Level 8] Finished running ['git', 'annex', 'version', '--raw'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'init', '--debug', '-c', 'annex.dotfiles=true'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'init', '--debug', '-c', 'annex.dotfiles=true'] [DEBUG ] Process 5639 started [DEBUG ] Waiting for process 5639 to complete [INFO ] [2021-06-21 17:09:10.891146224] process [5689] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","show-ref","--hash","refs/heads/git-annex"] [INFO ] [2021-06-21 17:09:12.150591638] process [5689] done ExitFailure 1 [INFO ] [2021-06-21 17:09:12.368376912] process [5700] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","show-ref","--verify","-q","origin/git-annex"] [INFO ] [2021-06-21 17:09:12.477195348] process [5700] done ExitFailure 1 [INFO ] [2021-06-21 17:09:12.751918525] process [5719] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","write-tree"] [INFO ] [2021-06-21 17:09:12.859730886] process [5719] done ExitSuccess [INFO ] [2021-06-21 17:09:12.861401936] process [5733] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","commit-tree","4b825dc642cb6eb9a060e54bf8d69288fbee4904","--no-gpg-sign"] [INFO ] [2021-06-21 17:09:13.352251412] process [5733] done ExitSuccess [INFO ] [2021-06-21 17:09:13.353326591] process [5740] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","update-ref","refs/heads/git-annex","74e6c54b490de653fdeee10f4b6758b8eae7d84b"] [INFO ] [2021-06-21 17:09:13.962687682] process [5740] done ExitSuccess [INFO ] [2021-06-21 17:09:14.127096295] process [5756] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","config","annex.uuid","a8204947-2249-48b4-9686-d5d91487fccc"] [INFO ] [2021-06-21 17:09:14.879866958] process [5756] done ExitSuccess [INFO ] [2021-06-21 17:09:14.881982298] process [5767] read: git ["config","--null","--list"] [INFO ] [2021-06-21 17:09:15.23901531] process [5767] done ExitSuccess [INFO ] [2021-06-21 17:09:15.310119778] process [5771] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","show-ref","git-annex"] [INFO ] [2021-06-21 17:09:16.002188003] process [5771] done ExitSuccess [INFO ] [2021-06-21 17:09:16.004470485] process [5776] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","show-ref","--hash","refs/heads/git-annex"] [INFO ] [2021-06-21 17:09:16.507840666] process [5776] done ExitSuccess [INFO ] [2021-06-21 17:09:16.511789865] process [5782] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","log","refs/heads/git-annex..74e6c54b490de653fdeee10f4b6758b8eae7d84b","--pretty=%H","-n1"] [INFO ] [2021-06-21 17:09:16.987330627] process [5782] done ExitSuccess [INFO ] [2021-06-21 17:09:16.990404623] process [5786] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","cat-file","--batch"] [INFO ] [2021-06-21 17:09:16.991353634] process [5787] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] [INFO ] [2021-06-21 17:09:17.745953471] process [5791] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","config","annex.version","8"] [INFO ] [2021-06-21 17:09:19.443639821] process [5791] done ExitSuccess [INFO ] [2021-06-21 17:09:19.446872726] process [5807] read: git ["config","--null","--list"] [INFO ] [2021-06-21 17:09:19.553415832] process [5807] done ExitSuccess [INFO ] [2021-06-21 17:09:19.555318385] process [5808] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","status","--porcelain"] [INFO ] [2021-06-21 17:09:20.401984886] process [5808] done ExitSuccess [INFO ] [2021-06-21 17:09:20.403586519] process [5822] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","config","filter.annex.smudge","git-annex smudge -- %f"] [INFO ] [2021-06-21 17:09:20.701595791] process [5822] done ExitSuccess [INFO ] [2021-06-21 17:09:20.704383308] process [5829] read: git ["config","--null","--list"] [INFO ] [2021-06-21 17:09:20.848635464] process [5829] done ExitSuccess [INFO ] [2021-06-21 17:09:20.850482365] process [5830] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","config","filter.annex.clean","git-annex smudge --clean -- %f"] [INFO ] [2021-06-21 17:09:21.365248336] process [5830] done ExitSuccess [INFO ] [2021-06-21 17:09:21.366493741] process [5832] read: git ["config","--null","--list"] [INFO ] [2021-06-21 17:09:21.760069711] process [5832] done ExitSuccess [INFO ] Scanning for unlocked files (this may take some time) [INFO ] [2021-06-21 17:09:21.792365226] process [5833] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","show-ref","--head"] [INFO ] [2021-06-21 17:09:22.101393837] process [5833] done ExitSuccess [INFO ] [2021-06-21 17:09:22.201169644] process [5838] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","symbolic-ref","-q","HEAD"] [INFO ] [2021-06-21 17:09:23.061983784] process [5838] done ExitSuccess [INFO ] [2021-06-21 17:09:23.064042537] process [5848] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","show-ref","refs/heads/master"] [INFO ] [2021-06-21 17:09:23.761411923] process [5848] done ExitFailure 1 [INFO ] [2021-06-21 17:09:23.763447524] process [5860] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","symbolic-ref","-q","HEAD"] [INFO ] [2021-06-21 17:09:23.87631655] process [5860] done ExitSuccess [INFO ] [2021-06-21 17:09:23.878164234] process [5861] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","show-ref","--hash","refs/heads/master"] [INFO ] [2021-06-21 17:09:24.296682792] process [5861] done ExitFailure 1 [INFO ] [2021-06-21 17:09:24.298134278] process [5869] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","checkout","-q","-B","master"] [INFO ] [2021-06-21 17:09:24.400538563] process [5869] done ExitSuccess [INFO ] [2021-06-21 17:09:24.690058862] process [5882] read: uname ["-n"] [INFO ] [2021-06-21 17:09:24.80420695] process [5882] done ExitSuccess [INFO ] [2021-06-21 17:09:25.315456594] process [5895] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","hash-object","-w","--stdin-paths","--no-filters"] [INFO ] [2021-06-21 17:09:25.316330308] process [5896] feed: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","update-index","-z","--index-info"] [INFO ] [2021-06-21 17:09:26.113931232] process [5896] done ExitSuccess [INFO ] [2021-06-21 17:09:26.115932496] process [5908] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","show-ref","--hash","refs/heads/git-annex"] [INFO ] [2021-06-21 17:09:26.702702852] process [5908] done ExitSuccess [INFO ] [2021-06-21 17:09:26.705521772] process [5920] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","write-tree"] [INFO ] [2021-06-21 17:09:27.223838903] process [5920] done ExitSuccess [INFO ] [2021-06-21 17:09:27.225662277] process [5929] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","commit-tree","e350cb3d76de8deff8de590012d636dff162291e","--no-gpg-sign","-p","refs/heads/git-annex"] [INFO ] [2021-06-21 17:09:27.857071371] process [5929] done ExitSuccess [INFO ] [2021-06-21 17:09:27.858376949] process [5935] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","update-ref","refs/heads/git-annex","2a257fafe1e5f4b9f31558d716432d806604ca2d"] [INFO ] [2021-06-21 17:09:28.185879989] process [5935] done ExitSuccess [INFO ] [2021-06-21 17:09:28.46275347] process [5786] done ExitSuccess [INFO ] [2021-06-21 17:09:28.463115358] process [5787] done ExitSuccess [INFO ] [2021-06-21 17:09:28.465044715] process [5895] done ExitSuccess [DEBUG ] Process 5639 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'init', '--debug', '-c', 'annex.dotfiles=true'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', 'config', '-z', '-l', '--show-origin'] [DEBUG ] Launching process ['git', 'config', '-z', '-l', '--show-origin'] [DEBUG ] Process 5965 started [DEBUG ] Waiting for process 5965 to complete [DEBUG ] Process 5965 exited with return code 0 [Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'check-attr', '-z', '--all', '--', '.'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'check-attr', '-z', '--all', '--', '.'] [DEBUG ] Process 5999 started [DEBUG ] Waiting for process 5999 to complete [DEBUG ] Process 5999 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'check-attr', '-z', '--all', '--', '.'] with status 0 [DEBUG ] Setting annex backend to MD5E (persistently) [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', 'config', '--local', 'annex.backends', 'MD5E'] [DEBUG ] Launching process ['git', 'config', '--local', 'annex.backends', 'MD5E'] [DEBUG ] Process 6033 started [DEBUG ] Waiting for process 6033 to complete [DEBUG ] Process 6033 exited with return code 0 [Level 8] Finished running ['git', 'config', '--local', 'annex.backends', 'MD5E'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', 'config', '-z', '-l', '--show-origin'] [DEBUG ] Launching process ['git', 'config', '-z', '-l', '--show-origin'] [DEBUG ] Process 6069 started [DEBUG ] Waiting for process 6069 to complete [DEBUG ] Process 6069 exited with return code 0 [Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', '-c', 'annex.largefiles=nothing', 'add', '--dry-run', '-N', '--ignore-missing', '--verbose', '--', '.gitattributes'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', '-c', 'annex.largefiles=nothing', 'add', '--dry-run', '-N', '--ignore-missing', '--verbose', '--', '.gitattributes'] [DEBUG ] Process 6134 started [DEBUG ] Waiting for process 6134 to complete [DEBUG ] Process 6134 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', '-c', 'annex.largefiles=nothing', 'add', '--dry-run', '-N', '--ignore-missing', '--verbose', '--', '.gitattributes'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', '-c', 'annex.largefiles=nothing', 'add', '--verbose', '--', '.gitattributes'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', '-c', 'annex.largefiles=nothing', 'add', '--verbose', '--', '.gitattributes'] [DEBUG ] Process 6172 started [DEBUG ] Waiting for process 6172 to complete [DEBUG ] Process 6172 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', '-c', 'annex.largefiles=nothing', 'add', '--verbose', '--', '.gitattributes'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'check-attr', '-z', '--all', '--', '.datalad/config', '.datalad/metadata/aggregate*', '.datalad/metadata/objects/'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'check-attr', '-z', '--all', '--', '.datalad/config', '.datalad/metadata/aggregate*', '.datalad/metadata/objects/*'] [DEBUG ] Process 6273 started [DEBUG ] Waiting for process 6273 to complete [DEBUG ] Process 6273 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'check-attr', '-z', '--all', '--', '.datalad/config', '.datalad/metadata/aggregate', '.datalad/metadata/objects/'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'check-attr', '-z', '--all', '--', '.git'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'check-attr', '-z', '--all', '--', '.git'] [DEBUG ] Process 6309 started [DEBUG ] Waiting for process 6309 to complete [DEBUG ] Process 6309 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'check-attr', '-z', '--all', '--', '.git'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', 'config', '--file', '/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.datalad/config', '--add', 'datalad.dataset.id', '95916503-503e-4bd7-bc37-a673f1bd0d1c'] [DEBUG ] Launching process ['git', 'config', '--file', '/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.datalad/config', '--add', 'datalad.dataset.id', '95916503-503e-4bd7-bc37-a673f1bd0d1c'] [DEBUG ] Process 6357 started [DEBUG ] Waiting for process 6357 to complete [DEBUG ] Process 6357 exited with return code 0 [Level 8] Finished running ['git', 'config', '--file', '/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.datalad/config', '--add', 'datalad.dataset.id', '95916503-503e-4bd7-bc37-a673f1bd0d1c'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', 'config', '-z', '-l', '--show-origin', '--file', '/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.datalad/config'] [DEBUG ] Launching process ['git', 'config', '-z', '-l', '--show-origin', '--file', '/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.datalad/config'] [DEBUG ] Process 6390 started [DEBUG ] Waiting for process 6390 to complete [DEBUG ] Process 6390 exited with return code 0 [Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin', '--file', '/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.datalad/config'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'diff', '--name-only', '--staged'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'diff', '--name-only', '--staged'] [DEBUG ] Process 6424 started [DEBUG ] Waiting for process 6424 to complete [DEBUG ] Process 6424 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'diff', '--name-only', '--staged'] with status 0 [DEBUG ] AnnexRepo(/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms).get_content_info(...) [DEBUG ] Query repo: ['ls-files', '--stage', '-z', '--exclude-standard', '-o'] [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z', '--exclude-standard', '-o', '--', '.datalad'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z', '--exclude-standard', '-o', '--', '.datalad'] [DEBUG ] Process 6462 started [DEBUG ] Waiting for process 6462 to complete [DEBUG ] Process 6462 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z', '--exclude-standard', '-o', '--', '.datalad'] with status 0 [DEBUG ] Done query repo: ['ls-files', '--stage', '-z', '--exclude-standard', '-o'] [DEBUG ] Done AnnexRepo(/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms).get_content_info(...) [DEBUG ] 2 path(s) to add to AnnexRepo(/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms) {'.gitattributes': {'type': 'file', 'state': 'untracked'}, '.datalad': {'type': 'directory', 'state': 'untracked'}} [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', '-c', 'annex.largefiles=nothing', 'add', '--verbose', '--', '.gitattributes', '.datalad'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', '-c', 'annex.largefiles=nothing', 'add', '--verbose', '--', '.gitattributes', '.datalad'] [DEBUG ] Process 6498 started [DEBUG ] Waiting for process 6498 to complete [DEBUG ] Process 6498 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', '-c', 'annex.largefiles=nothing', 'add', '--verbose', '--', '.gitattributes', '.datalad'] with status 0 [DEBUG ] Committing via direct call of git: ['git', '-c', 'diff.ignoreSubmodules=none', 'commit', '-m', '[DATALAD] new dataset'] [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] [DEBUG ] Process 6721 started [DEBUG ] Waiting for process 6721 to complete [DEBUG ] Process 6721 exited with return code 1 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] with status 1 [Level 11] CommandError: 'git -c diff.ignoreSubmodules=none rev-parse --quiet --verify 'HEAD^{commit}'' failed with exitcode 1 under /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'commit', '-m', '[DATALAD] new dataset', '--', '.gitattributes', '.datalad'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'commit', '-m', '[DATALAD] new dataset', '--', '.gitattributes', '.datalad'] [DEBUG ] Process 6759 started [DEBUG ] Waiting for process 6759 to complete [DEBUG ] Process 6759 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'commit', '-m', '[DATALAD] new dataset', '--', '.gitattributes', '.datalad'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'symbolic-ref', 'HEAD'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'symbolic-ref', 'HEAD'] [DEBUG ] Process 6971 started [DEBUG ] Waiting for process 6971 to complete [DEBUG ] Process 6971 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'symbolic-ref', 'HEAD'] with status 0 [DEBUG ] No sync necessary, no corresponding branch detected [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'symbolic-ref', 'HEAD'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'symbolic-ref', 'HEAD'] [DEBUG ] Process 7002 started [DEBUG ] Waiting for process 7002 to complete [DEBUG ] Process 7002 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'symbolic-ref', 'HEAD'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'checkout', '-b', 'incoming'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'checkout', '-b', 'incoming'] [DEBUG ] Process 7034 started [DEBUG ] Waiting for process 7034 to complete [DEBUG ] Process 7034 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'checkout', '-b', 'incoming'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'initremote', 'datalad-archives', 'encryption=none', 'type=external', 'externaltype=datalad-archives', 'autoenable=true', 'uuid=c04eb54b-4b4e-5755-8436-866b043170fa', '--debug', '-c', 'annex.dotfiles=true', '-c', 'annex.retry=3'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'initremote', 'datalad-archives', 'encryption=none', 'type=external', 'externaltype=datalad-archives', 'autoenable=true', 'uuid=c04eb54b-4b4e-5755-8436-866b043170fa', '--debug', '-c', 'annex.dotfiles=true', '-c', 'annex.retry=3'] [DEBUG ] Process 7094 started [DEBUG ] Waiting for process 7094 to complete [DEBUG ] Process 7094 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'initremote', 'datalad-archives', 'encryption=none', 'type=external', 'externaltype=datalad-archives', 'autoenable=true', 'uuid=c04eb54b-4b4e-5755-8436-866b043170fa', '--debug', '-c', 'annex.dotfiles=true', '-c', 'annex.retry=3'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', 'config', '-z', '-l', '--show-origin'] [DEBUG ] Launching process ['git', 'config', '-z', '-l', '--show-origin'] [DEBUG ] Process 7516 started [DEBUG ] Waiting for process 7516 to complete [DEBUG ] Process 7516 exited with return code 0 [Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', '-c', 'annex.largefiles=nothing', 'add', '--dry-run', '-N', '--ignore-missing', '--verbose', '--', 'T1_MPRAGE_COR_ND_0008.tar.gz'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', '-c', 'annex.largefiles=nothing', 'add', '--dry-run', '-N', '--ignore-missing', '--verbose', '--', 'T1_MPRAGE_COR_ND_0008.tar.gz'] [DEBUG ] Process 7554 started [DEBUG ] Waiting for process 7554 to complete [DEBUG ] Process 7554 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', '-c', 'annex.largefiles=nothing', 'add', '--dry-run', '-N', '--ignore-missing', '--verbose', '--', 'T1_MPRAGE_COR_ND_0008.tar.gz'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'add', '--json', '--json-error-messages', '--debug', '-c', 'annex.dotfiles=true', '-c', 'annex.retry=3', '--', 'T1_MPRAGE_COR_ND_0008.tar.gz'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'add', '--json', '--json-error-messages', '--debug', '-c', 'annex.dotfiles=true', '-c', 'annex.retry=3', '--', 'T1_MPRAGE_COR_ND_0008.tar.gz'] [DEBUG ] Process 7586 started [DEBUG ] Waiting for process 7586 to complete
[DEBUG ] Process 7586 exited with return code 0
[Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'add', '--json', '--json-error-messages', '--debug', '-c', 'annex.dotfiles=true', '-c', 'annex.retry=3', '--', 'T1_MPRAGE_COR_ND_0008.tar.gz'] with status 0 [DEBUG ] Committing via direct call of git: ['git', '-c', 'diff.ignoreSubmodules=none', 'commit', '-m', 'Retrieved ../../sub-test/ses-1/T1_MPRAGE_COR_ND_0008.tar.gz'] [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] [DEBUG ] Process 7789 started [DEBUG ] Waiting for process 7789 to complete [DEBUG ] Process 7789 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'commit', '-m', 'Retrieved ../../sub-test/ses-1/T1_MPRAGE_COR_ND_0008.tar.gz', '--'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'commit', '-m', 'Retrieved ../../sub-test/ses-1/T1_MPRAGE_COR_ND_0008.tar.gz', '--'] [DEBUG ] Process 7821 started [DEBUG ] Waiting for process 7821 to complete [DEBUG ] Process 7821 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'commit', '-m', 'Retrieved ../../sub-test/ses-1/T1_MPRAGE_COR_ND_0008.tar.gz', '--'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'checkout', '--orphan', 'incoming-processed'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'checkout', '--orphan', 'incoming-processed'] [DEBUG ] Process 7958 started [DEBUG ] Waiting for process 7958 to complete [DEBUG ] Process 7958 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'checkout', '--orphan', 'incoming-processed'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'status', '--porcelain', '--untracked-files=normal', '--ignore-submodules=none'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'status', '--porcelain', '--untracked-files=normal', '--ignore-submodules=none'] [DEBUG ] Process 8046 started [DEBUG ] Waiting for process 8046 to complete [DEBUG ] Process 8046 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'status', '--porcelain', '--untracked-files=normal', '--ignore-submodules=none'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] [DEBUG ] Process 8081 started [DEBUG ] Waiting for process 8081 to complete [DEBUG ] Process 8081 exited with return code 1 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] with status 1 [Level 11] CommandError: 'git -c diff.ignoreSubmodules=none rev-parse --quiet --verify 'HEAD^{commit}'' failed with exitcode 1 under /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms [DEBUG ] AnnexRepo(/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms).get_content_info(...) [DEBUG ] Query repo: ['ls-files', '--stage', '-z', '--exclude-standard', '-o', '--directory', '--no-empty-directory'] [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z', '--exclude-standard', '-o', '--directory', '--no-empty-directory'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z', '--exclude-standard', '-o', '--directory', '--no-empty-directory'] [DEBUG ] Process 8133 started [DEBUG ] Waiting for process 8133 to complete [DEBUG ] Process 8133 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z', '--exclude-standard', '-o', '--directory', '--no-empty-directory'] with status 0 [DEBUG ] Done query repo: ['ls-files', '--stage', '-z', '--exclude-standard', '-o', '--directory', '--no-empty-directory'] [DEBUG ] Done AnnexRepo(/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms).get_content_info(...) [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '-z', '-m'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '-z', '-m'] [DEBUG ] Process 8168 started [DEBUG ] Waiting for process 8168 to complete [DEBUG ] Process 8168 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '-z', '-m'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'rm', '-f', '-r', '--', '.'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'rm', '-f', '-r', '--', '.'] [DEBUG ] Process 8233 started [DEBUG ] Waiting for process 8233 to complete [DEBUG ] Process 8233 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'rm', '-f', '-r', '--', '.'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'merge', '-s', 'ours', '--no-commit', '--allow-unrelated-histories', 'incoming'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'merge', '-s', 'ours', '--no-commit', '--allow-unrelated-histories', 'incoming'] [DEBUG ] Process 8297 started [DEBUG ] Waiting for process 8297 to complete [DEBUG ] Process 8297 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'merge', '-s', 'ours', '--no-commit', '--allow-unrelated-histories', 'incoming'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'read-tree', '-m', '-u', 'incoming'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'read-tree', '-m', '-u', 'incoming'] [DEBUG ] Process 8425 started [DEBUG ] Waiting for process 8425 to complete [DEBUG ] Process 8425 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'read-tree', '-m', '-u', 'incoming'] with status 0 [DEBUG ] Building doc for <class 'datalad.local.subdatasets.Subdatasets'> [DEBUG ] Building doc for <class 'datalad.core.distributed.clone.Clone'> [DEBUG ] Building doc for <class 'datalad.distribution.get.Get'> [DEBUG ] Building doc for <class 'datalad.distribution.install.Install'> [DEBUG ] Building doc for <class 'datalad.distribution.publish.Publish'> [DEBUG ] Building doc for <class 'datalad.core.distributed.push.Push'> [DEBUG ] Building doc for <class 'datalad.distribution.drop.Drop'> [DEBUG ] Building doc for <class 'datalad.distribution.uninstall.Uninstall'> [DEBUG ] Building doc for <class 'datalad.distribution.remove.Remove'> [DEBUG ] Building doc for <class 'datalad.distribution.update.Update'> [DEBUG ] Building doc for <class 'datalad.distribution.siblings.Siblings'> [DEBUG ] Building doc for <class 'datalad.distribution.create_sibling.CreateSibling'> [DEBUG ] Building doc for <class 'datalad.distribution.create_sibling_github.CreateSiblingGithub'> [DEBUG ] Building doc for <class 'datalad.distributed.create_sibling_gitlab.CreateSiblingGitlab'> [DEBUG ] Building doc for <class 'datalad.distributed.create_sibling_ria.CreateSiblingRia'> [DEBUG ] Building doc for <class 'datalad.interface.unlock.Unlock'> [DEBUG ] Building doc for <class 'datalad.local.copy_file.CopyFile'> [DEBUG ] Building doc for <class 'datalad.metadata.metadata.Metadata'> [DEBUG ] Building doc for <class 'datalad.metadata.search.Search'> [DEBUG ] Building doc for <class 'datalad.metadata.aggregate.AggregateMetaData'> [DEBUG ] Building doc for <class 'datalad.metadata.extract_metadata.ExtractMetadata'> [DEBUG ] Building doc for <class 'datalad.plugin.wtf.WTF'> [DEBUG ] Building doc for <class 'datalad.interface.test.Test'> [DEBUG ] Building doc for <class 'datalad.interface.ls.Ls'> [DEBUG ] Building doc for <class 'datalad.interface.clean.Clean'> [DEBUG ] Building doc for <class 'datalad.interface.add_archive_content.AddArchiveContent'> [DEBUG ] Building doc for <class 'datalad.interface.shell_completion.ShellCompletion'> [DEBUG ] Building doc for <class 'datalad.core.local.run.Run'> [DEBUG ] Building doc for <class 'datalad.interface.rerun.Rerun'> [DEBUG ] Building doc for <class 'datalad.interface.run_procedure.RunProcedure'> [DEBUG ] Building doc for <class 'datalad.distributed.export_archive_ora.ExportArchiveORA'> [DEBUG ] Building doc for <class 'datalad.distribution.create_test_dataset.CreateTestDataset'> [DEBUG ] Building doc for <class 'datalad.support.sshrun.SSHRun'> [DEBUG ] Query status of AnnexRepo('/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms') for all paths [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] [DEBUG ] Process 8576 started [DEBUG ] Waiting for process 8576 to complete [DEBUG ] Process 8576 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'rev-parse', '--quiet', '--verify', 'HEAD^{commit}'] with status 0 [DEBUG ] AnnexRepo(/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms).get_content_info(...) [DEBUG ] Query repo: ['ls-files', '--stage', '-z', '--exclude-standard', '-o'] [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z', '--exclude-standard', '-o'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z', '--exclude-standard', '-o'] [DEBUG ] Process 8613 started [DEBUG ] Waiting for process 8613 to complete [DEBUG ] Process 8613 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '--stage', '-z', '--exclude-standard', '-o'] with status 0 [DEBUG ] Done query repo: ['ls-files', '--stage', '-z', '--exclude-standard', '-o'] [DEBUG ] Done AnnexRepo(/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms).get_content_info(...) [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '-z', '-m'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '-z', '-m'] [DEBUG ] Process 8648 started [DEBUG ] Waiting for process 8648 to complete [DEBUG ] Process 8648 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-files', '-z', '-m'] with status 0 [DEBUG ] AnnexRepo(/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms).get_content_info(...) [DEBUG ] Query repo: ['ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l'] [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l'] [DEBUG ] Process 8685 started [DEBUG ] Waiting for process 8685 to complete [DEBUG ] Process 8685 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l'] with status 0 [DEBUG ] Done query repo: ['ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l'] [DEBUG ] Initiating a new process for BatchedCommand(cmd=['git', 'cat-file', '--batch'], output_proc=, path=<<'/work/fatx405++50 chars++oms'>>) [DEBUG ] Closing stderr of <subprocess.Popen object at 0x146aac1e8590> [DEBUG ] Closing stdin of <subprocess.Popen object at 0x146aac1e8590> and waiting process to finish [DEBUG ] Process <subprocess.Popen object at 0x146aac1e8590> has finished [DEBUG ] Done AnnexRepo(/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms).get_content_info(...) [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'lookupkey', '--debug', '-c', 'annex.dotfiles=true', '-c', 'annex.retry=3', '--', 'T1_MPRAGE_COR_ND_0008.tar.gz'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'lookupkey', '--debug', '-c', 'annex.dotfiles=true', '-c', 'annex.retry=3', '--', 'T1_MPRAGE_COR_ND_0008.tar.gz'] [DEBUG ] Process 8773 started [DEBUG ] Waiting for process 8773 to complete [DEBUG ] Process 8773 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'lookupkey', '--debug', '-c', 'annex.dotfiles=true', '-c', 'annex.retry=3', '--', 'T1_MPRAGE_COR_ND_0008.tar.gz'] with status 0 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'contentlocation', 'MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz', '--debug', '-c', 'annex.dotfiles=true', '-c', 'annex.retry=3'] [DEBUG ] Launching process ['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'contentlocation', 'MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz', '--debug', '-c', 'annex.dotfiles=true', '-c', 'annex.retry=3'] [DEBUG ] Process 8845 started [DEBUG ] Waiting for process 8845 to complete [DEBUG ] Process 8845 exited with return code 0 [Level 8] Finished running ['git', '-c', 'diff.ignoreSubmodules=none', 'annex', 'contentlocation', 'MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz', '--debug', '-c', 'annex.dotfiles=true', '-c', 'annex.retry=3'] with status 0 [INFO ] Adding content of the archive T1_MPRAGE_COR_ND_0008.tar.gz into annex AnnexRepo(/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms) [DEBUG ] Async run: | cwd=None | cmd=['7z'] [DEBUG ] Launching process ['7z'] [DEBUG ] Process 8915 started [DEBUG ] Waiting for process 8915 to complete [DEBUG ] Process 8915 exited with return code 0 [Level 8] Finished running ['7z'] with status 0 [DEBUG ] Not initiating existing cache for the archives under /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/datalad/tmp/archives [DEBUG ] Not initiating existing cache for the archives under /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/datalad/tmp/archives [DEBUG ] Cached directory for archive /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/annex/objects/90/zf/MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz/MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz is ad77b92bc5 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms | cmd=['git', 'config', '-z', '-l', '--show-origin', '--file', '/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.datalad/config'] [DEBUG ] Launching process ['git', 'config', '-z', '-l', '--show-origin', '--file', '/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.datalad/config'] [DEBUG ] Process 8918 started [DEBUG ] Waiting for process 8918 to complete [DEBUG ] Process 8918 exited with return code 0 [Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin', '--file', '/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.datalad/config'] with status 0 [DEBUG ] Special remote datalad-archives already exists [DEBUG ] Acquiring a lock /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/datalad/tmp/archives/ad77b92bc5.extract-lck [DEBUG ] Acquired? lock /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/datalad/tmp/archives/ad77b92bc5.extract-lck: True [DEBUG ] Extracting /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/annex/objects/90/zf/MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz/MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz under /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/datalad/tmp/archives/ad77b92bc5 [DEBUG ] Async run: | cwd=/work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/datalad/tmp/archives/ad77b92bc5 | cmd=7z x /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/annex/objects/90/zf/MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz/MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz -so | 7z x -si -ttar [DEBUG ] Launching process 7z x /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/annex/objects/90/zf/MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz/MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz -so | 7z x -si -ttar [DEBUG ] Process 8957 started [DEBUG ] Waiting for process 8957 to complete [DEBUG ] Process 8957 exited with return code 0 [Level 8] Finished running '7z x /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/annex/objects/90/zf/MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz/MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz -so | 7z x -si -ttar' with status 0 [DEBUG ] Releasing lock /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/datalad/tmp/archives/ad77b92bc5.extract-lck [DEBUG ] Adding /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/T1_MPRAGE_COR_ND_0008/DCM_102.MR.STUDIEN_NEUROLOGIE.0008.0188.2018.12.18.12.07.27.214150.163607951.IMA to annex pointing to dl+archive:MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz#path=T1_MPRAGE_COR_ND_0008/DCM_102.MR.STUDIEN_NEUROLOGIE.0008.0188.2018.12.18.12.07.27.214150.163607951.IMA&size=287232 and with options None [DEBUG ] Initiating a new process for BatchedAnnex(cmd=<<['git', '-c', ++117 chars++ug']>>, output_proc=, path=<<'/work/fatx405++50 chars++oms'>>) [DEBUG ] Closing stderr of <subprocess.Popen object at 0x146aac17f4d0> [DEBUG ] Closing stdin of <subprocess.Popen object at 0x146aac17f4d0> and waiting process to finish [DEBUG ] Process <subprocess.Popen object at 0x146aac17f4d0> has finished [DEBUG ] Cleaning up the cache for /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/annex/objects/90/zf/MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz/MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz under /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/datalad/tmp/archives/ad77b92bc5 [DEBUG ] Cleaning up the stamp file for /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/annex/objects/90/zf/MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz/MD5E-s72842081--349c00e692b5449e33e0138f9cb644f4.tar.gz under /work/fatx405/projects/2106_test_proc/sourcedata/001_struct/dicoms/.git/datalad/tmp/archives/ad77b92bc5.stamp

yarikoptic commented 3 years ago

output of datalad wtf -D html_detail might end up of help

yarikoptic commented 3 years ago 
Failed to fetch any archive containing URL-s287232--dl,43archive:MD5E-s72842081--34-71d3f8069a40ce1960d8450c16d73b15

might be the leading culprit... so -- some original archive was added in annex add --fast mode?

bpoldrack commented 3 years ago

@m-petersen

That looks like you are trying to add that archive to dicoms subdataset, that was created by import-dcm from another archive before. That's not (yet) supported. The intention for that command is to produce such a subdataset from a single archive.

There are several ways to deal with that. Let me know whether you need help. Ultimately you are not bound to the dataset structure suggested by that command. It's just meant to support a structure that we thought useful.

m-petersen commented 3 years ago

Dear @yarikoptic, dear @bpoldrack,

thanks a lot for your reply and help. Since I need a working solution bidsification and anonymization soon I moved to implementing heudiconv directly. By the way facing another error (https://github.com/nipy/heudiconv/issues/516).

That looks like you are trying to add that archive to dicoms subdataset, that was created by import-dcm from another archive before

I reproduced the error with datalad-hirni when creating a new dataset without previously added archives.

some original archive was added in annex add --fast mode?

The only commands I performed were datalad create test_hirni cd test_hirni datalad run-procedure cfg_hirni datalad hirni-import-dcm ../testdata_ewgenia_tar/ewgenia001/ses-1/EP2D_DIFF_96DIR_7B0_3SHELL_ORIG_0015.tar.gz

output of datalad wtf -D html_detail might end up of help

WTF

configuration <SENSITIVE, report disabled by configuration>

credentials

  • keyring:
  • active_backends:
  • PlaintextKeyring with no encyption v.1.0 at /home/fatx405/.local/share/python_keyring/keyring_pass.cfg
  • config_file: /home/fatx405/.config/python_keyring/keyringrc.cfg
  • data_root: /home/fatx405/.local/share/python_keyring

    datalad

  • full_version: 0.14.4
  • version: 0.14.4

    dependencies

  • annexremote: 1.5.0
  • appdirs: 1.4.4
  • boto: 2.49.0
  • cmd:7z: 16.02
  • cmd:annex: 8.20201104-g13bab4f2c
  • cmd:bundled-git: 2.29.2
  • cmd:git: 2.29.2
  • cmd:system-git: 2.29.2
  • cmd:system-ssh: 7.4p1
  • exifread: 2.1.2
  • humanize: 3.2.0
  • iso8601: 0.1.14
  • keyring: 22.0.1
  • keyrings.alt: 4.0.2
  • msgpack: 1.0.2
  • mutagen: 1.41.1
  • requests: 2.25.1
  • wrapt: 1.12.1

    environment

  • LANG: en_US.UTF-8
  • PATH: /work/fatx405/miniconda3/bin:/sw/link/git/2.21.0/bin:/sw/env/system-gcc/singularity/3.5.2-overlayfix/bin:/sw/batch/slurm/19.05.6/bin:/sw/rrz/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

    extensions

  • container:
  • description: Containerized environments
  • entrypoints:
  • datalad_container.containers_add.ContainersAdd:
  • class: ContainersAdd
  • load_error: None
  • module: datalad_container.containers_add
  • names:
  • containers-add
  • containers_add
  • datalad_container.containers_list.ContainersList:
  • class: ContainersList
  • load_error: None
  • module: datalad_container.containers_list
  • names:
  • containers-list
  • containers_list
  • datalad_container.containers_remove.ContainersRemove:
  • class: ContainersRemove
  • load_error: None
  • module: datalad_container.containers_remove
  • names:
  • containers-remove
  • containers_remove
  • datalad_container.containers_run.ContainersRun:
  • class: ContainersRun
  • load_error: None
  • module: datalad_container.containers_run
  • names:
  • containers-run
  • containers_run
  • load_error: None
  • module: datalad_container
  • version: 1.1.4
  • hirni:
  • description: HIRNI workflows
  • entrypoints:
  • datalad_hirni.commands.dicom2spec.Dicom2Spec:
  • class: Dicom2Spec
  • load_error: None
  • module: datalad_hirni.commands.dicom2spec
  • names:
  • hirni-dicom2spec
  • hirni_dicom2spec
  • datalad_hirni.commands.import_dicoms.ImportDicoms:
  • class: ImportDicoms
  • load_error: None
  • module: datalad_hirni.commands.import_dicoms
  • names:
  • hirni-import-dcm
  • hirni_import_dcm
  • datalad_hirni.commands.spec2bids.Spec2Bids:
  • class: Spec2Bids
  • load_error: None
  • module: datalad_hirni.commands.spec2bids
  • names:
  • hirni-spec2bids
  • hirni_spec2bids
  • datalad_hirni.commands.spec4anything.Spec4Anything:
  • class: Spec4Anything
  • load_error: None
  • module: datalad_hirni.commands.spec4anything
  • names:
  • hirni-spec4anything
  • hirni_spec4anything
  • load_error: None
  • module: datalad_hirni
  • version: 0.0.8
  • metalad:
  • description: DataLad semantic metadata command suite
  • entrypoints:
  • datalad_metalad.aggregate.Aggregate:
  • class: Aggregate
  • load_error: None
  • module: datalad_metalad.aggregate
  • names:
  • meta-aggregate
  • meta_aggregate
  • datalad_metalad.dump.Dump:
  • class: Dump
  • load_error: None
  • module: datalad_metalad.dump
  • names:
  • meta-dump
  • meta_dump
  • datalad_metalad.extract.Extract:
  • class: Extract
  • load_error: None
  • module: datalad_metalad.extract
  • names:
  • meta-extract
  • meta_extract
  • load_error: None
  • module: datalad_metalad
  • version: 0.2.1
  • neuroimaging:
  • description: Neuroimaging tools
  • entrypoints:
  • datalad_neuroimaging.bids2scidata.BIDS2Scidata:
  • class: BIDS2Scidata
  • load_error: None
  • module: datalad_neuroimaging.bids2scidata
  • names:
  • bids2scidata
  • load_error: None
  • module: datalad_neuroimaging
  • version: 0.3.1
  • ukbiobank:
  • description: UKBiobank dataset support
  • entrypoints:
  • datalad_ukbiobank.init.Init:
  • class: Init
  • load_error: None
  • module: datalad_ukbiobank.init
  • names:
  • ukb-init
  • ukb_init
  • datalad_ukbiobank.update.Update:
  • class: Update
  • load_error: None
  • module: datalad_ukbiobank.update
  • names:
  • ukb-update
  • ukb_update
  • load_error: None
  • module: datalad_ukbiobank
  • version: 0.3.2
  • webapp:
  • description: Generic web app support
  • entrypoints:
  • datalad_webapp.WebApp:
  • class: WebApp
  • load_error: None
  • module: datalad_webapp
  • names:
  • webapp
  • webapp
  • load_error: None
  • module: datalad_webapp
  • version: 0.3

    git-annex

  • build flags:
  • Assistant
  • Webapp
  • Pairing
  • Inotify
  • DBus
  • DesktopNotify
  • TorrentParser
  • MagicMime
  • Feeds
  • Testsuite
  • S3
  • WebDAV
  • dependency versions:
  • aws-0.22
  • bloomfilter-2.0.1.0
  • cryptonite-0.26
  • DAV-1.3.4
  • feed-1.3.0.1
  • ghc-8.8.4
  • http-client-0.6.4.1
  • persistent-sqlite-2.10.6.2
  • torrent-10000.1.1
  • uuid-1.3.13
  • yesod-1.6.1.0
  • key/value backends:
  • SHA256E
  • SHA256
  • SHA512E
  • SHA512
  • SHA224E
  • SHA224
  • SHA384E
  • SHA384
  • SHA3_256E
  • SHA3_256
  • SHA3_512E
  • SHA3_512
  • SHA3_224E
  • SHA3_224
  • SHA3_384E
  • SHA3_384
  • SKEIN256E
  • SKEIN256
  • SKEIN512E
  • SKEIN512
  • BLAKE2B256E
  • BLAKE2B256
  • BLAKE2B512E
  • BLAKE2B512
  • BLAKE2B160E
  • BLAKE2B160
  • BLAKE2B224E
  • BLAKE2B224
  • BLAKE2B384E
  • BLAKE2B384
  • BLAKE2BP512E
  • BLAKE2BP512
  • BLAKE2S256E
  • BLAKE2S256
  • BLAKE2S160E
  • BLAKE2S160
  • BLAKE2S224E
  • BLAKE2S224
  • BLAKE2SP256E
  • BLAKE2SP256
  • BLAKE2SP224E
  • BLAKE2SP224
  • SHA1E
  • SHA1
  • MD5E
  • MD5
  • WORM
  • URL
  • X*
  • operating system: linux x86_64
  • remote types:
  • git
  • gcrypt
  • p2p
  • S3
  • bup
  • directory
  • rsync
  • web
  • bittorrent
  • webdav
  • adb
  • tahoe
  • glacier
  • ddar
  • git-lfs
  • httpalso
  • hook
  • external
  • supported repository versions:
  • 8
  • upgrade supported from repository versions:
  • 0
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • version: 8.20201104-g13bab4f2c

    location

  • path: /home/fatx405
  • type: directory

    metadata_extractors

  • annex (datalad 0.14.4):
  • distribution: datalad 0.14.4
  • load_error: None
  • module: datalad.metadata.extractors.annex
  • version: None
  • audio (datalad 0.14.4):
  • distribution: datalad 0.14.4
  • load_error: None
  • module: datalad.metadata.extractors.audio
  • version: None
  • bids (datalad-neuroimaging 0.3.1):
  • distribution: datalad-neuroimaging 0.3.1
  • load_error: None
  • module: datalad_neuroimaging.extractors.bids
  • version: None
  • datacite (datalad 0.14.4):
  • distribution: datalad 0.14.4
  • load_error: None
  • module: datalad.metadata.extractors.datacite
  • version: None
  • datalad_core (datalad 0.14.4):
  • distribution: datalad 0.14.4
  • load_error: None
  • module: datalad.metadata.extractors.datalad_core
  • version: None
  • datalad_rfc822 (datalad 0.14.4):
  • distribution: datalad 0.14.4
  • load_error: None
  • module: datalad.metadata.extractors.datalad_rfc822
  • version: None
  • dicom (datalad-neuroimaging 0.3.1):
  • distribution: datalad-neuroimaging 0.3.1
  • load_error: None
  • module: datalad_neuroimaging.extractors.dicom
  • version: None
  • exif (datalad 0.14.4):
  • distribution: datalad 0.14.4
  • load_error: None
  • module: datalad.metadata.extractors.exif
  • version: None
  • frictionless_datapackage (datalad 0.14.4):
  • distribution: datalad 0.14.4
  • load_error: None
  • module: datalad.metadata.extractors.frictionless_datapackage
  • version: None
  • image (datalad 0.14.4):
  • distribution: datalad 0.14.4
  • load_error: None
  • module: datalad.metadata.extractors.image
  • version: None
  • metalad_annex (datalad-metalad 0.2.1):
  • distribution: datalad-metalad 0.2.1
  • load_error: None
  • module: datalad_metalad.extractors.annex
  • version: None
  • metalad_core (datalad-metalad 0.2.1):
  • distribution: datalad-metalad 0.2.1
  • load_error: None
  • module: datalad_metalad.extractors.core
  • version: None
  • metalad_custom (datalad-metalad 0.2.1):
  • distribution: datalad-metalad 0.2.1
  • load_error: None
  • module: datalad_metalad.extractors.custom
  • version: None
  • metalad_runprov (datalad-metalad 0.2.1):
  • distribution: datalad-metalad 0.2.1
  • load_error: None
  • module: datalad_metalad.extractors.runprov
  • version: None
  • nidm (datalad-neuroimaging 0.3.1):
  • distribution: datalad-neuroimaging 0.3.1
  • load_error: None
  • module: datalad_neuroimaging.extractors.nidm
  • version: None
  • nifti1 (datalad-neuroimaging 0.3.1):
  • distribution: datalad-neuroimaging 0.3.1
  • load_error: None
  • module: datalad_neuroimaging.extractors.nifti1
  • version: None
  • xmp (datalad 0.14.4):
  • distribution: datalad 0.14.4
  • load_error: Exempi library not found. [exempi.py:_load_exempi:60]
  • module: datalad.metadata.extractors.xmp

    metadata_indexers

    python

  • implementation: CPython
  • version: 3.7.9

    system

  • distribution: CentOS Linux/7.9.2009/Core
  • encoding:
  • default: utf-8
  • filesystem: utf-8
  • locale.prefered: UTF-8
  • max_path_length: 269
  • name: Linux
  • release: 4.14.237-1.0.32.el7.rrz.x86_64
  • type: posix
  • version: #1 SMP Thu Jun 17 19:41:45 CEST 2021