Closed lsempf closed 3 years ago
Hello @lsempf,
there's nothing that strikes me as an incredibly obvious problem. At a first glance, I'd suspect that something went wrong with the umlaut encoding. When you say "other MRI measurements could be imported", does that include within-archive paths with an umlaut?
I'll try to reproduce it, but the issue might be completely elsewhere, of course. If you have a chance to rerun that very command by adding -l 8
right after datalad
, I'd be very interested in its output.
Hi @bpoldrack,
thank you for your response! Since we added this particular measurement later, the other acquisitions didn’t have an 'umlaut'.
So, I tried this command: datalad -l 8 hirni-import-dcm --anon-subject 001 /subject_001/subject_001_dicomsorted_nr0001.tar subject_001
I got a lot of [DEBUG ] messages (see at the end) but the command ended with an error:
[DEBUG ] Cleaning up the cache for /subject_001/dicoms/.git/annex/objects/G9/64/MD5E-s2886256640--0b6a7b53982a09e98d1ad0ace2b95655.tar/MD5E-s2886256640--0b6a7b53982a09e98d1ad0ace2b95655.tar under /subject_001/dicoms/.git/datalad/tmp/archives/06238fc134 [DEBUG ] Cleaning up the stamp file for /subject_001/dicoms/.git/annex/objects/G9/64/MD5E-s2886256640--0b6a7b53982a09e98d1ad0ace2b95655.tar/MD5E-s2886256640--0b6a7b53982a09e98d1ad0ace2b95655.tar under /subject_001/dicoms/.git/datalad/tmp/archives/06238fc134.stamp AnnexBatchCommandError: command 'addurl' Error, annex reported failure for addurl (url='dl+archive:MD5E-s2886256640--0b6a7b53982a09e98d1ad0ace2b95655.tar#path=au70_3841/study_1_20200622/series_5_Lautst%C3%A4rketest_Schichtfg.egal_SBRef/MR.1.3.12.2.1107.5.2.43.66017.2020062210180192305602876&size=1785482'): {'command': 'addurl', 'note': 'from datalad-archives\nto subject_001/study_1_20200622/series_5_Lautstärketest_Schichtfg.egal_SBRef/MR.1.3.12.2.1107.5.2.43.66017.2020062210180192305602876', 'success': False, 'error-messages': [], 'file': 'subject_001/study_1_20200622/series_5_Lautstärketest_Schichtfg.egal_SBRef/MR.1.3.12.2.1107.5.2.43.66017.2020062210180192305602876'}
Here is the complete output:
[DEBUG ] Command line args 1st pass. Parsed: Namespace() Unparsed: ['hirni-import-dcm', '--anon-subject', '032', '/home/data/sfb_cuetarget/scratch/mini_helios/au70_3841/au70_3841_dicomsorted_nr6680.tar', 'au70_3841']
[DEBUG ] Discovering plugins
[DEBUG ] Loading entrypoints
[DEBUG ] Loading entrypoint webapp from datalad.extensions for docs building
[DEBUG ] Building doc for <class 'datalad_webapp.WebApp'>
[DEBUG ] Loaded entrypoint webapp
[DEBUG ] Loading entrypoint neuroimaging from datalad.extensions for docs building
[DEBUG ] Loaded entrypoint neuroimaging
[DEBUG ] Loading entrypoint metalad from datalad.extensions for docs building
[DEBUG ] Loaded entrypoint metalad
[DEBUG ] Loading entrypoint hirni from datalad.extensions for docs building
[DEBUG ] Loaded entrypoint hirni
[DEBUG ] Loading entrypoint container from datalad.extensions for docs building
[DEBUG ] Loaded entrypoint container
[DEBUG ] Building doc for <class 'datalad.core.local.status.Status'>
[DEBUG ] Building doc for <class 'datalad.core.local.save.Save'>
[DEBUG ] Building doc for <class 'datalad.interface.annotate_paths.AnnotatePaths'>
[DEBUG ] Building doc for <class 'datalad_metalad.dump.Dump'>
[DEBUG ] Building doc for <class 'datalad_hirni.commands.spec4anything.Spec4Anything'>
[DEBUG ] Building doc for <class 'datalad_hirni.commands.dicom2spec.Dicom2Spec'>
[DEBUG ] Building doc for <class 'datalad_metalad.extract.Extract'>
[DEBUG ] Building doc for <class 'datalad.core.local.diff.Diff'>
[DEBUG ] Building doc for <class 'datalad_metalad.aggregate.Aggregate'>
[DEBUG ] Building doc for <class 'datalad.interface.download_url.DownloadURL'>
[DEBUG ] Building doc for <class 'datalad_hirni.commands.import_dicoms.ImportDicoms'>
[DEBUG ] Parsing known args among ['/home/data/sfb_cuetarget/env/datalad_12_5/bin/datalad', '-l', '8', 'hirni-import-dcm', '--anon-subject', '032', '/home/data/sfb_cuetarget/scratch/mini_helios/au70_3841/au70_3841_dicomsorted_nr6680.tar', 'au70_3841']
[Level 9] Running: ['git', '--git-dir=', 'config', '-z', '-l', '--show-origin']
[Level 8] Finished running ['git', '--git-dir=', 'config', '-z', '-l', '--show-origin'] with status 0
[DEBUG ] Determined class of decorated function: <class 'datalad_hirni.commands.import_dicoms.ImportDicoms'>
[DEBUG ] Resolved dataset for import DICOM session: /home/data/sfb_cuetarget/sourcedata
[Level 9] Running: ['git', '--git-dir=', 'config', '-z', '-l', '--show-origin']
[Level 8] Finished running ['git', '--git-dir=', 'config', '-z', '-l', '--show-origin'] with status 0
[Level 9] Running: ['git', 'config', '-z', '-l', '--show-origin', '--file', '/home/data/sfb_cuetarget/sourcedata/.datalad/config']
[Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin', '--file', '/home/data/sfb_cuetarget/sourcedata/.datalad/config'] with status 0
[Level 9] Running: ['git', 'config', '-z', '-l', '--show-origin']
[Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0
[DEBUG ] Discovering plugins
[DEBUG ] Building doc for <class 'datalad.core.local.create.Create'>
[DEBUG ] Found matching interface ('datalad.core.local.create', 'Create') for create
[DEBUG ] Determined class of decorated function: <class 'datalad.core.local.create.Create'>
[DEBUG ] Query status of <GitRepo path=/home/data/sfb_cuetarget/sourcedata (<class 'datalad.support.gitrepo.GitRepo'>)> for 1 paths
[Level 9] Running: ['git', 'show', '-z', '--no-patch', '--format=%H', '--']
[Level 8] Finished running ['git', 'show', '-z', '--no-patch', '--format=%H', '--'] with status 0
[DEBUG ] <GitRepo path=/home/data/sfb_cuetarget/sourcedata (<class 'datalad.support.gitrepo.GitRepo'>)>.get_content_info(...)
[Level 8] Using existing Git repository at /home/data/sfb_cuetarget/sourcedata
[Level 9] Running: ['git', 'config', '-z', '-l', '--file', '.gitmodules']
[Level 8] Finished running ['git', 'config', '-z', '-l', '--file', '.gitmodules'] with status 0
[DEBUG ] <GitRepo path=/home/data/sfb_cuetarget/sourcedata (<class 'datalad.support.gitrepo.GitRepo'>)>.get_content_info(...)
[DEBUG ] Query repo: ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard']
[Level 9] Running: ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard']
[Level 8] Finished running ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard'] with status 0
[DEBUG ] Done query repo: ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard']
[DEBUG ] Done <GitRepo path=/home/data/sfb_cuetarget/sourcedata (<class 'datalad.support.gitrepo.GitRepo'>)>.get_content_info(...)
[DEBUG ] Query repo: ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard']
[Level 9] Running: ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard', '--', 'au70_3841/dicoms']
[Level 8] Finished running ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard', '--', 'au70_3841/dicoms'] with status 0
[DEBUG ] Done query repo: ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard']
[DEBUG ] Done <GitRepo path=/home/data/sfb_cuetarget/sourcedata (<class 'datalad.support.gitrepo.GitRepo'>)>.get_content_info(...)
[Level 9] Running: ['git', 'ls-files', '-z', '-m', '--', 'au70_3841/dicoms']
[Level 8] Finished running ['git', 'ls-files', '-z', '-m', '--', 'au70_3841/dicoms'] with status 0
[DEBUG ] <GitRepo path=/home/data/sfb_cuetarget/sourcedata (<class 'datalad.support.gitrepo.GitRepo'>)>.get_content_info(...)
[DEBUG ] Query repo: ['git', 'ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l']
[Level 9] Running: ['git', 'ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l', '--', 'au70_3841/dicoms']
[Level 8] Finished running ['git', 'ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l', '--', 'au70_3841/dicoms'] with status 0
[DEBUG ] Done query repo: ['git', 'ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l']
[DEBUG ] Done <GitRepo path=/home/data/sfb_cuetarget/sourcedata (<class 'datalad.support.gitrepo.GitRepo'>)>.get_content_info(...)
[INFO ] Creating a new annex repo at /home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms
[DEBUG ] Initialize empty Git repository at '/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms'
[Level 9] Running: ['git', 'init']
[Level 8] Finished running ['git', 'init'] with status 0
[Level 9] Running: ['git', 'for-each-ref', '--format=%(refname:strip=2)', 'refs/heads', 'refs/remotes']
[Level 8] Finished running ['git', 'for-each-ref', '--format=%(refname:strip=2)', 'refs/heads', 'refs/remotes'] with status 0
[DEBUG ] Initializing annex repository at /home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms...
[Level 9] Running: ['git', 'config', '-z', '-l', '--show-origin']
[Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0
[Level 9] Running: ['git', 'config', '-z', '-l', '--show-origin']
[Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0
[Level 9] Running: ['git', 'for-each-ref', '--format=%(refname:strip=2)', 'refs/heads']
[Level 8] Finished running ['git', 'for-each-ref', '--format=%(refname:strip=2)', 'refs/heads'] with status 0
[Level 9] Running: ['git', 'annex', 'version', '--raw']
[Level 8] Finished running ['git', 'annex', 'version', '--raw'] with status 0
[Level 9] Running: ['git-annex', 'init', '-c', 'annex.dotfiles=true', '--debug']
[INFO ] [2020-10-06 12:27:26.714482017] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","show-ref","--hash","refs/heads/git-annex"]
[INFO ] [2020-10-06 12:27:26.721636531] process done ExitFailure 1
[INFO ] [2020-10-06 12:27:26.721910589] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","show-ref","--verify","-q","origin/git-annex"]
[INFO ] [2020-10-06 12:27:26.727599035] process done ExitFailure 1
[INFO ] [2020-10-06 12:27:26.739507266] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","write-tree"]
[INFO ] [2020-10-06 12:27:26.824099147] process done ExitSuccess
[INFO ] [2020-10-06 12:27:26.824286778] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","commit-tree","4b825dc642cb6eb9a060e54bf8d69288fbee4904","--no-gpg-sign"]
[INFO ] [2020-10-06 12:27:26.882277732] process done ExitSuccess
[INFO ] [2020-10-06 12:27:26.882483741] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","update-ref","refs/heads/git-annex","d73cbc764d26deb280d07cae708d4f03f2fd6a97"]
[INFO ] [2020-10-06 12:27:26.963224557] process done ExitSuccess
[INFO ] [2020-10-06 12:27:26.998991134] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","config","annex.uuid","83f054ba-8e77-4c88-a3bb-714acc956bf1"]
[INFO ] [2020-10-06 12:27:27.040822084] process done ExitSuccess
[INFO ] [2020-10-06 12:27:27.041053829] read: git ["config","--null","--list"]
[INFO ] [2020-10-06 12:27:27.049473616] process done ExitSuccess
[INFO ] [2020-10-06 12:27:27.254458609] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","show-ref","git-annex"]
[INFO ] [2020-10-06 12:27:27.264184415] process done ExitSuccess
[INFO ] [2020-10-06 12:27:27.264357372] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","show-ref","--hash","refs/heads/git-annex"]
[INFO ] [2020-10-06 12:27:27.272377953] process done ExitSuccess
[INFO ] [2020-10-06 12:27:27.274055259] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","log","refs/heads/git-annex..d73cbc764d26deb280d07cae708d4f03f2fd6a97","--pretty=%H","-n1"]
[INFO ] [2020-10-06 12:27:27.283125628] process done ExitSuccess
[INFO ] [2020-10-06 12:27:27.284409338] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","cat-file","--batch"]
[INFO ] [2020-10-06 12:27:27.285513686] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
[INFO ] [2020-10-06 12:27:27.292016824] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","config","annex.version","5"]
[INFO ] [2020-10-06 12:27:27.33236521] process done ExitSuccess
[INFO ] [2020-10-06 12:27:27.332529557] read: git ["config","--null","--list"]
[INFO ] [2020-10-06 12:27:27.341521639] process done ExitSuccess
[INFO ] [2020-10-06 12:27:27.341720642] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","-c","annex.dotfiles=true","symbolic-ref","-q","HEAD"]
[INFO ] [2020-10-06 12:27:27.347458259] process done ExitSuccess
[INFO ] [2020-10-06 12:27:27.347949452] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","-c","annex.dotfiles=true","show-ref","refs/heads/master"]
[INFO ] [2020-10-06 12:27:27.355106797] process done ExitFailure 1
[INFO ] [2020-10-06 12:27:27.355275203] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","-c","annex.dotfiles=true","symbolic-ref","-q","HEAD"]
[INFO ] [2020-10-06 12:27:27.361396416] process done ExitSuccess
[INFO ] [2020-10-06 12:27:27.361544008] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","-c","annex.dotfiles=true","show-ref","--hash","refs/heads/master"]
[INFO ] [2020-10-06 12:27:27.368257531] process done ExitFailure 1
[INFO ] [2020-10-06 12:27:27.368413223] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","-c","annex.dotfiles=true","checkout","-q","-B","master"]
[INFO ] [2020-10-06 12:27:27.430956364] process done ExitSuccess
[INFO ] [2020-10-06 12:27:27.563365888] read: uname ["-n"]
[INFO ] [2020-10-06 12:27:27.566845344] process done ExitSuccess
[INFO ] [2020-10-06 12:27:27.719818416] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","-c","annex.dotfiles=true","hash-object","-w","--stdin-paths","--no-filters"]
[INFO ] [2020-10-06 12:27:27.721540274] feed: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","-c","annex.dotfiles=true","update-index","-z","--index-info"]
[INFO ] [2020-10-06 12:27:27.87251846] process done ExitSuccess
[INFO ] [2020-10-06 12:27:27.872743646] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","-c","annex.dotfiles=true","show-ref","--hash","refs/heads/git-annex"]
[INFO ] [2020-10-06 12:27:27.880196406] process done ExitSuccess
[INFO ] [2020-10-06 12:27:27.881419659] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","-c","annex.dotfiles=true","write-tree"]
[INFO ] [2020-10-06 12:27:27.964088006] process done ExitSuccess
[INFO ] [2020-10-06 12:27:27.96426649] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","-c","annex.dotfiles=true","commit-tree","308f75998081d3ccb0c3dd0c929888bab2ccfece","--no-gpg-sign","-p","refs/heads/git-annex"]
[INFO ] [2020-10-06 12:27:28.022582796] process done ExitSuccess
[INFO ] [2020-10-06 12:27:28.022770477] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.dotfiles=true","-c","annex.dotfiles=true","-c","annex.dotfiles=true","update-ref","refs/heads/git-annex","4ef5d1fd8f637a66a06f635cf2d783af0e414e6e"]
[INFO ] [2020-10-06 12:27:28.064200292] process done ExitSuccess
[INFO ] [2020-10-06 12:27:28.125062027] process done ExitSuccess
[INFO ] [2020-10-06 12:27:28.126016009] process done ExitSuccess
[INFO ] [2020-10-06 12:27:28.127373885] process done ExitSuccess
[Level 8] Finished running ['git-annex', 'init', '-c', 'annex.dotfiles=true', '--debug'] with status 0
[Level 9] Running: ['git', 'config', '-z', '-l', '--show-origin']
[Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0
[Level 9] Running: ['git', 'config', '-z', '-l', '--show-origin']
[Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0
[Level 9] Running: ['git', 'check-attr', '-z', '--all', '--', '.']
[Level 8] Finished running ['git', 'check-attr', '-z', '--all', '--', '.'] with status 0
[DEBUG ] Setting annex backend to MD5E (persistently)
[Level 9] Running: ['git', 'config', '--local', 'annex.backends', 'MD5E']
[Level 8] Finished running ['git', 'config', '--local', 'annex.backends', 'MD5E'] with status 0
[Level 9] Running: ['git', 'config', '-z', '-l', '--show-origin']
[Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0
[Level 9] Running: ['git', '-c', 'annex.largefiles=nothing', 'add', '--dry-run', '-N', '--ignore-missing', '--verbose', '--', '.gitattributes']
[Level 8] Finished running ['git', '-c', 'annex.largefiles=nothing', 'add', '--dry-run', '-N', '--ignore-missing', '--verbose', '--', '.gitattributes'] with status 0
[Level 9] Running: ['git-annex', 'add', '-c', 'annex.dotfiles=true', '--json', '--json-error-messages', '-c', 'annex.largefiles=nothing', '--include-dotfiles', '--debug', '--', '.gitattributes']
[Level 8] Finished running ['git-annex', 'add', '-c', 'annex.dotfiles=true', '--json', '--json-error-messages', '-c', 'annex.largefiles=nothing', '--include-dotfiles', '--debug', '--', '.gitattributes'] with status 0
[Level 9] Running: ['git', 'check-attr', '-z', '--all', '--', '.datalad/config', '.datalad/metadata/aggregate*', '.datalad/metadata/objects/*']
[Level 8] Finished running ['git', 'check-attr', '-z', '--all', '--', '.datalad/config', '.datalad/metadata/aggregate', '.datalad/metadata/objects/**'] with status 0
[Level 9] Running: ['git', 'check-attr', '-z', '--all', '--', '.git']
[Level 8] Finished running ['git', 'check-attr', '-z', '--all', '--', '.git'] with status 0
[Level 9] Running: ['git', 'config', '--file', '/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms/.datalad/config', '--add', 'datalad.dataset.id', '891a2840-07be-11eb-997d-a0369f7c647e']
[Level 8] Finished running ['git', 'config', '--file', '/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms/.datalad/config', '--add', 'datalad.dataset.id', '891a2840-07be-11eb-997d-a0369f7c647e'] with status 0
[Level 9] Running: ['git', 'config', '-z', '-l', '--show-origin', '--file', '/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms/.datalad/config']
[Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin', '--file', '/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms/.datalad/config'] with status 0
[Level 9] Running: ['git', 'config', '-z', '-l', '--show-origin']
[Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0
[Level 9] Running: ['git', 'diff', '--name-only', '--staged']
[Level 8] Finished running ['git', 'diff', '--name-only', '--staged'] with status 0
[DEBUG ] <AnnexRepo path=/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms (<class 'datalad.support.annexrepo.AnnexRepo'>)>.get_content_info(...)
[Level 8] Using existing Git repository at /home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms
[DEBUG ] Query repo: ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard', '-o']
[Level 9] Running: ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard', '-o', '--', '.datalad']
[Level 8] Finished running ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard', '-o', '--', '.datalad'] with status 0
[DEBUG ] Done query repo: ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard', '-o']
[DEBUG ] Done <AnnexRepo path=/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms (<class 'datalad.support.annexrepo.AnnexRepo'>)>.get_content_info(...)
[DEBUG ] 2 path(s) to add to <AnnexRepo path=/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms (<class 'datalad.support.annexrepo.AnnexRepo'>)> {'.gitattributes': {'type': 'file', 'state': 'untracked'}, '.datalad': {'type': 'directory', 'state': 'untracked'}}
Total: 0%| | 0.00/59.0 [00:00<?, ?B/s][Level 9] Running: ['git-annex', 'add', '-c', 'annex.dotfiles=true', '--json', '--json-error-messages', '--include-dotfiles', '-c', 'annex.largefiles=nothing', '--debug', '--', '.gitattributes', '.datalad']
Total (3 ok out of 2): 93%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████▊ | 55.0/59.0 [00:00<00:00, 694B/s][Level 8] Finished running ['git-annex', 'add', '-c', 'annex.dotfiles=true', '--json', '--json-error-messages', '--include-dotfiles', '-c', 'annex.largefiles=nothing', '--debug', '--', '.gitattributes', '.datalad'] with status 0
[DEBUG ] Committing via direct call of git: ['git', 'commit', '-m', '[DATALAD] new dataset']
[Level 9] Running: ['git', 'commit', '-m', '[DATALAD] new dataset', '--', '.gitattributes', '.datalad']
[Level 8] Finished running ['git', 'commit', '-m', '[DATALAD] new dataset', '--', '.gitattributes', '.datalad'] with status 0
[Level 9] Running: ['git', 'symbolic-ref', 'HEAD']
[Level 8] Finished running ['git', 'symbolic-ref', 'HEAD'] with status 0
[Level 9] Running: ['git', 'checkout', '-b', 'incoming']
[Level 8] Finished running ['git', 'checkout', '-b', 'incoming'] with status 0
[DEBUG ] Reloading config due to supposed working tree update
[Level 9] Running: ['git', 'config', '-z', '-l', '--show-origin', '--file', '/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms/.datalad/config']
[Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin', '--file', '/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms/.datalad/config'] with status 0
[Level 9] Running: ['git', 'config', '-z', '-l', '--show-origin']
[Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0
[Level 9] Running: ['git-annex', 'initremote', '-c', 'annex.dotfiles=true', 'datalad-archives', 'encryption=none', 'type=external', 'externaltype=datalad-archives', 'autoenable=true', 'uuid=c04eb54b-4b4e-5755-8436-866b043170fa', '--debug']
[Level 8] Finished running ['git-annex', 'initremote', '-c', 'annex.dotfiles=true', 'datalad-archives', 'encryption=none', 'type=external', 'externaltype=datalad-archives', 'autoenable=true', 'uuid=c04eb54b-4b4e-5755-8436-866b043170fa', '--debug'] with status 0
[Level 9] Running: ['git', 'config', '-z', '-l', '--show-origin', '--file', '/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms/.datalad/config']
[Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin', '--file', '/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms/.datalad/config'] with status 0
[Level 9] Running: ['git', 'config', '-z', '-l', '--show-origin']
[Level 8] Finished running ['git', 'config', '-z', '-l', '--show-origin'] with status 0
[Level 9] Running: ['git', '-c', 'annex.largefiles=nothing', 'add', '--dry-run', '-N', '--ignore-missing', '--verbose', '--', 'au70_3841_dicomsorted_nr6680.tar']
[Level 8] Finished running ['git', '-c', 'annex.largefiles=nothing', 'add', '--dry-run', '-N', '--ignore-missing', '--verbose', '--', 'au70_3841_dicomsorted_nr6680.tar'] with status 0
[Level 9] Running: ['git-annex', 'add', '-c', 'annex.dotfiles=true', '--json', '--json-error-messages', '--debug', '--', 'au70_3841_dicomsorted_nr6680.tar']
[Level 8] Finished running ['git-annex', 'add', '-c', 'annex.dotfiles=true', '--json', '--json-error-messages', '--debug', '--', 'au70_3841_dicomsorted_nr6680.tar'] with status 0
[DEBUG ] Committing via direct call of git: ['git', 'commit', '-m', 'Retrieved /home/data/sfb_cuetarget/scratch/mini_helios/au70_3841/au70_3841_dicomsorted_nr6680.tar']
[Level 9] Running: ['git', 'commit', '-m', 'Retrieved /home/data/sfb_cuetarget/scratch/mini_helios/au70_3841/au70_3841_dicomsorted_nr6680.tar']
[Level 8] Finished running ['git', 'commit', '-m', 'Retrieved /home/data/sfb_cuetarget/scratch/mini_helios/au70_3841/au70_3841_dicomsorted_nr6680.tar'] with status 0
[Level 9] Running: ['git', 'checkout', '--orphan', 'incoming-processed']
[Level 8] Finished running ['git', 'checkout', '--orphan', 'incoming-processed'] with status 0
[DEBUG ] Reloading config due to supposed working tree update
[Level 9] Running: ['git', 'status', '--porcelain', '--untracked-files=normal', '--ignore-submodules=none']
[Level 8] Finished running ['git', 'status', '--porcelain', '--untracked-files=normal', '--ignore-submodules=none'] with status 0
[Level 9] Running: ['git', 'rm', '-f', '-r', '--', '.']
[Level 8] Finished running ['git', 'rm', '-f', '-r', '--', '.'] with status 0
[Level 9] Running: ['git', 'merge', '-s', 'ours', '--no-commit', 'incoming']
[Level 8] Finished running ['git', 'merge', '-s', 'ours', '--no-commit', 'incoming'] with status 0
[Level 9] Running: ['git', 'read-tree', '-m', '-u', 'incoming']
[Level 8] Finished running ['git', 'read-tree', '-m', '-u', 'incoming'] with status 0
[DEBUG ] Building doc for <class 'datalad.local.subdatasets.Subdatasets'>
[DEBUG ] Building doc for <class 'datalad.core.distributed.clone.Clone'>
[DEBUG ] Building doc for <class 'datalad.distribution.get.Get'>
[DEBUG ] Building doc for <class 'datalad.distribution.install.Install'>
[DEBUG ] Building doc for <class 'datalad.interface.save.Save'>
[DEBUG ] Building doc for <class 'datalad.distribution.add.Add'>
[DEBUG ] Building doc for <class 'datalad.distribution.publish.Publish'>
[DEBUG ] Building doc for <class 'datalad.distribution.drop.Drop'>
[DEBUG ] Building doc for <class 'datalad.distribution.uninstall.Uninstall'>
[DEBUG ] Building doc for <class 'datalad.distribution.remove.Remove'>
[DEBUG ] Building doc for <class 'datalad.distribution.update.Update'>
[DEBUG ] Building doc for <class 'datalad.distribution.siblings.Siblings'>
[DEBUG ] Building doc for <class 'datalad.distribution.create_sibling.CreateSibling'>
[DEBUG ] Building doc for <class 'datalad.distribution.create_sibling_github.CreateSiblingGithub'>
[DEBUG ] Building doc for <class 'datalad.distributed.create_sibling_gitlab.CreateSiblingGitlab'>
[DEBUG ] Building doc for <class 'datalad.interface.unlock.Unlock'>
[DEBUG ] Building doc for <class 'datalad.metadata.metadata.Metadata'>
[DEBUG ] Building doc for <class 'datalad.metadata.search.Search'>
[DEBUG ] Building doc for <class 'datalad.metadata.aggregate.AggregateMetaData'>
[DEBUG ] Building doc for <class 'datalad.metadata.extract_metadata.ExtractMetadata'>
[DEBUG ] Building doc for <class 'datalad.plugin.wtf.WTF'>
[DEBUG ] Building doc for <class 'datalad.interface.test.Test'>
[DEBUG ] Building doc for <class 'datalad.interface.ls.Ls'>
[DEBUG ] Building doc for <class 'datalad.interface.clean.Clean'>
[DEBUG ] Building doc for <class 'datalad.interface.add_archive_content.AddArchiveContent'>
[DEBUG ] Building doc for <class 'datalad.core.local.run.Run'>
[DEBUG ] Building doc for <class 'datalad.interface.rerun.Rerun'>
[DEBUG ] Building doc for <class 'datalad.interface.run_procedure.RunProcedure'>
[DEBUG ] Building doc for <class 'datalad.distribution.create_test_dataset.CreateTestDataset'>
[DEBUG ] Building doc for <class 'datalad.support.sshrun.SSHRun'>
[DEBUG ] Query status of <AnnexRepo path=/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms (<class 'datalad.support.annexrepo.AnnexRepo'>)> for all paths
[Level 9] Running: ['git', 'show', '-z', '--no-patch', '--format=%H', '--']
[Level 8] Finished running ['git', 'show', '-z', '--no-patch', '--format=%H', '--'] with status 0
[DEBUG ] <AnnexRepo path=/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms (<class 'datalad.support.annexrepo.AnnexRepo'>)>.get_content_info(...)
[DEBUG ] Query repo: ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard', '-o']
[Level 9] Running: ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard', '-o']
[Level 8] Finished running ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard', '-o'] with status 0
[DEBUG ] Done query repo: ['git', 'ls-files', '--stage', '-z', '-d', '-m', '--exclude-standard', '-o']
[DEBUG ] Done <AnnexRepo path=/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms (<class 'datalad.support.annexrepo.AnnexRepo'>)>.get_content_info(...)
[Level 9] Running: ['git', 'ls-files', '-z', '-m']
[Level 8] Finished running ['git', 'ls-files', '-z', '-m'] with status 0
[DEBUG ] <AnnexRepo path=/home/data/sfb_cuetarget/sourcedata/au70_3841/dicoms (<class 'datalad.support.annexrepo.AnnexRepo'>)>.get_content_info(...)
[DEBUG ] Query repo: ['git', 'ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l']
[Level 9] Running: ['git', 'ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l']
[Level 8] Finished running ['git', 'ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l'] with status 0
[DEBUG ] Done query repo: ['git', 'ls-tree', 'HEAD', '-z', '-r', '--full-tree', '-l']
[DEBUG ] Initiating a new process for BatchedCommand(cmd=['git', 'cat-file', '--batch'], output_proc=
Thanks, @lsempf !
I tried to reproduce the issue, assuming it was caused by the umlaut, but failed. So, that doesn't appear to be the issue. I assume, I can't have the tarball to look into it for data protection reasons or is there a way to share it?
Edit:
An alternative could be to build (and possibly share) an archive that only contains the file in question (see error message) including its path as you have it in the original archive and try to import this. If - as I assume - the error persists and that would make this miniature archive shareable, I could have a closer look.
Since git-annex sadly doesn't correctly report the actual error message within its JSON response, I need to have the issue right in front of me, I guess. A remote session to look into it might be another way.
Otherwise we need to ping-pong in this issue, where I tell you what to execute and you report the output. That could turn into quite a hassle. If it's fine with you, we can take that road, of course.
Update to keep the record in this issue: I tried to use your tarball and it worked just fine for me, using hirni in version 0.0.6 as well as the brandnew 0.0.7 So, this looks like an issue that's about your environment. Something like you are running out of space on disc and it's not properly reported.
Ok, so turns out it's about the umlaut indeed, leading to /344
being written as part of the path/URL on this particular machine.
Not actually an issue with datalad-hirni, thus - closing.
Hello!
I tried to import an MRI measurement with datalad hirni-import-dcm. The following error occurred:
We suspect that it is a git annex bug. Other MRI measurements could be imported without errors.
Thanks for any help.