datalad create -c hirni led to error

[TobiasKadelka]

I am just trying to create a hirni-dataset, but the cfg_hirni procedure gives me an error. I updated datalad this week to version "datalad 0.12.0rc6", so I wouldn't be surprised, if that might be the reason for the new error.

❱ datalad create -c hirni test_dataset                                                                                                                                                                       130 !
[INFO   ] Creating a new annex repo at /data/BnB1/Datalad/test_dataset 
[INFO   ] Running procedure cfg_hirni                                                                                                                                                                              
[INFO   ] == Command start (output follows) ===== 
[WARNING] Found no aggregated metadata info file /data/BnB1/Datalad/test_dataset/.datalad/metadata/aggregate_v1.json. You will likely need to either update the dataset from its original location or reaggregate metadata locally. 
[WARNING] Dataset at . contains no aggregated metadata on this path [metadata(/data/BnB1/Datalad/test_dataset)] 
[WARNING] Could not obtain dataset metadata, proceeding without 
[INFO   ] Cloning https://github.com/psychoinformatics-de/hirni-toolbox.git [1 other candidates] into '/data/BnB1/Datalad/test_dataset/code/hirni-toolbox' 
[INFO   ]   Remote origin not usable by git-annex; setting annex-ignore                                                                                                                                            
Traceback (most recent call last):                                                                                                                                                                                 
  File "/home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad_hirni/resources/procedures/cfg_hirni.py", line 94, in <module>                                                                     
    "procedure-call": "bash {script} {{location}} {ds}/README"                                                                                                                                                     
  File "/home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad/distribution/dataset.py", line 497, in apply_func
    return f(**kwargs)
  File "/home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad/interface/utils.py", line 485, in eval_func
    return return_func(generator_func)(*args, **kwargs)
  File "/home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad/interface/utils.py", line 473, in return_func
    results = list(results)
  File "/home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad/interface/utils.py", line 421, in generator_func
    result_log_level, **_kwargs):
  File "/home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad/interface/utils.py", line 516, in _process_results
    for res in results:
  File "/home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad_hirni/commands/spec4anything.py", line 162, in __call__
    on_failure='ignore',
  File "/home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad/interface/utils.py", line 421, in generator_func
    result_log_level, **_kwargs):
  File "/home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad/interface/utils.py", line 516, in _process_results
    for res in results:
  File "/home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad/interface/annotate_paths.py", line 616, in __call__
    path = rawpath2ap(path, refds_path)
  File "/home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad/interface/annotate_paths.py", line 313, in rawpath2ap
    path = _resolve_path(path, refds_path)
  File "/home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad/interface/annotate_paths.py", line 77, in _resolve_path
    path = expandpath(path, force_absolute=False)
  File "/home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad/utils.py", line 328, in expandpath
    path = expandvars(expanduser(path))
  File "/home/homeGlobal/tkadelka/env/hirni/lib/python3.5/posixpath.py", line 229, in expanduser
    if not path.startswith(tilde):
AttributeError: 'PosixPath' object has no attribute 'startswith'
[INFO   ] == Command exit (modification check follows) ===== 
CommandError: command '/home/homeGlobal/tkadelka/env/hirni/bin/python3 /home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad_hirni/resources/procedures/cfg_hirni.py /data/BnB1/Datalad/test_dataset ' failed with exitcode 1
Failed to run '/home/homeGlobal/tkadelka/env/hirni/bin/python3 /home/homeGlobal/tkadelka/env/hirni/lib/python3.5/site-packages/datalad_hirni/resources/procedures/cfg_hirni.py /data/BnB1/Datalad/test_dataset ' under '/data/BnB1/Datalad/test_dataset'. Exit code=1.

[TobiasKadelka]

A little addition: when I let the hirni-script run, it will try to add Files with "datalad hirni-import-dcm" which leads to this error:

[INFO   ] Finished adding T10660.tar: Files processed: 1644, +annex: 1644 
[ERROR  ] argument should be string, bytes or integer, not PosixPath [genericpath.py:exists:19] (TypeError)                                                                                                        
[INFO   ] Creating a new annex repo at /data/BnB1/Datalad/DTA_study/.git/datalad/hirni_import/dicoms                                                                                                               

This might be because of the missing hirni configuration?

[bpoldrack]

Thanks for reporting, @TobiasKadelka . That looks like an incompatibility with rc6, that we missed indeed! I'll fix that and make a hirni release. Meanwhile you might want to use just another environment for datalad-hirni, that would use rc5 instead.

[bpoldrack]

FTR: I could reproduce it on brainbfast with an rc6 environment, but failed to do so locally. There's something else to it.

[bpoldrack]

I think it's the python version. datalad 0.12.0rc6 + python 3.5 + datalad-hirni is the problem.

[pvavra]

I also get that error with datalad 0.12.0rc6 and python 3.5

with python 3.6, this error goes away, but a related error (https://github.com/psychoinformatics-de/hirni-toolbox/issues/8) occurs at a later stage, when working through the demo steps in the docs.

the 3.6 error is not (yet) there with datalad 0.12.0rc5, but is there in 0.12.0rc6 -- in case this helps hunting this down..