search

psychoinformatics-de / datalad-hirni

DataLad extension for (semi-)automated, reproducible processing of (medical/neuro)imaging data
http://datalad.org
Other
5 stars 8 forks source link

Seemingly needless metadata aggregation/use in/for spec4anything #116

Closed mih closed 5 years ago

mih commented 5 years ago

cfg_hirni runs metadata aggregation-- unclear why. If removed, it crashes, because a meta_dump() call in spec4anything fails (no meta anymore).

Traceback (most recent call last):
  File "/home/mih/hacking/datalad/hirni/datalad_hirni/resources/procedures/cfg_hirni.py", line 94, in <module>
    "procedure-call": "bash {script} {{location}} {ds}/README"
  File "/home/mih/hacking/datalad/git/datalad/distribution/dataset.py", line 525, in apply_func
    return f(**kwargs)
  File "/home/mih/hacking/datalad/git/datalad/interface/utils.py", line 491, in eval_func
    return return_func(generator_func)(*args, **kwargs)
  File "/home/mih/hacking/datalad/git/datalad/interface/utils.py", line 479, in return_func
    results = list(results)
  File "/home/mih/hacking/datalad/git/datalad/interface/utils.py", line 428, in generator_func
    result_renderer, result_xfm, _result_filter, **_kwargs):
  File "/home/mih/hacking/datalad/git/datalad/interface/utils.py", line 520, in _process_results
    for res in results:
  File "/home/mih/hacking/datalad/hirni/datalad_hirni/commands/spec4anything.py", line 146, in __call__
    result_renderer='disabled')

Looking for why metadata is accessed, I can only see metadata access for the dataset ID (no need to aggregate metadata for that), and for refcommit. The latter is never read for anything and seemingly only placed in the spec FTR.

I propose to stop metadata aggregation in cfg_hirni and adjust the remaining commands to now access metadata when not needed. This would improve the user experience (lots of implication-less warnings gone), make things faster, and nothing is lost AFAICS.