Open jsheunis opened 2 years ago
With both meta-extract and meta-conduct, if they're run with the metalad_studyminimeta extractor on a dataset that doesn't have a .studyminimeta.yaml file locally available, the error messages are not informative.
meta-extract
meta-conduct
metalad_studyminimeta
.studyminimeta.yaml
Example 1:
>> datalad meta-extract -d /Users/jsheunis/Documents/psyinf/Data/super metalad_studyminimeta Studyminimeta metadata extraction: 0%|
Example 2:
>> datalad -f json meta-extract -d /Users/jsheunis/Documents/psyinf/Data/super metalad_studyminimeta {"action": "meta_extract", "message": "file /Users/jsheunis/Documents/psyinf/Data/super/.studyminimeta.yaml could not be opened", "path": "/Users/jsheunis/Documents/psyinf/Data/super", "status": "error", "type": "dataset"} Studyminimeta metadata extraction: 0%|
Example 3:
>> datalad -f json meta-conduct /Users/jsheunis/Documents/psyinf/datalad-catalog/datalad_catalog/examples/extract_dataset_metadata.json \ traverser:/Users/jsheunis/Documents/psyinf/Data/super \ traverser:dataset \ extractor:Dataset \ extractor:metalad_studyminimeta Studyminimeta metadata extraction: 0%| | 0.00/37.0 [00:00<?, ? Files/s][ERROR ] Exception None. 1 failed: | [{'action': 'meta_extract', | 'message': 'file ' | '/Users/jsheunis/Documents/psyinf/Data/super/.studyminimeta.yaml ' | 'could not be opened', | 'path': '/Users/jsheunis/Documents/psyinf/Data/super', | 'status': 'error', | 'type': 'dataset'}] in processor <Future at 0x7fd73e5fa910 state=finished raised IncompleteResultsError> {"action": "meta_conduct", "message": "concurrent.futures.process._RemoteTraceback: \n\"\"\"\nTraceback (most recent call last):\n File \"/Users/jsheunis/opt/miniconda3/envs/catalognew/lib/python3.9/concurrent/futures/process.py\", line 243, in _process_worker\n r = call_item.fn(*call_item.args, **call_item.kwargs)\n File \"/Users/jsheunis/opt/miniconda3/envs/catalognew/lib/python3.9/site-packages/datalad_metalad/processor/base.py\", line 26, in execute\n return context, self.process(pipeline_element)\n File \"/Users/jsheunis/opt/miniconda3/envs/catalognew/lib/python3.9/site-packages/datalad_metalad/processor/extract.py\", line 85, in process\n for extract_result in meta_extract(**kwargs):\n File \"/Users/jsheunis/opt/miniconda3/envs/catalognew/lib/python3.9/site-packages/datalad/interface/utils.py\", line 483, in eval_func\n return return_func(generator_func)(*args, **kwargs)\n File \"/Users/jsheunis/opt/miniconda3/envs/catalognew/lib/python3.9/site-packages/datalad/interface/utils.py\", line 476, in return_func\n results = list(results)\n File \"/Users/jsheunis/opt/miniconda3/envs/catalognew/lib/python3.9/site-packages/datalad/interface/utils.py\", line 461, in generator_func\n raise IncompleteResultsError(\ndatalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:\n[{'action': 'meta_extract',\n 'message': 'file '\n '/Users/jsheunis/Documents/psyinf/Data/super/.studyminimeta.yaml '\n 'could not be opened',\n 'path': '/Users/jsheunis/Documents/psyinf/Data/super',\n 'status': 'error',\n 'type': 'dataset'}]\n\"\"\"\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/Users/jsheunis/opt/miniconda3/envs/catalognew/lib/python3.9/site-packages/datalad_metalad/conduct.py\", line 281, in process_parallel\n source_index, pipeline_element = future.result()\n File \"/Users/jsheunis/opt/miniconda3/envs/catalognew/lib/python3.9/concurrent/futures/_base.py\", line 438, in result\n return self.__get_result()\n File \"/Users/jsheunis/opt/miniconda3/envs/catalognew/lib/python3.9/concurrent/futures/_base.py\", line 390, in __get_result\n raise self._exception\ndatalad.support.exceptions.IncompleteResultsError: None. 1 failed:\n[{'action': 'meta_extract',\n 'message': 'file '\n '/Users/jsheunis/Documents/psyinf/Data/super/.studyminimeta.yaml '\n 'could not be opened',\n 'path': '/Users/jsheunis/Documents/psyinf/Data/super',\n 'status': 'error',\n 'type': 'dataset'}]\n", "status": "error"}
Or to rephrase, the informative parts of the error messages are nested deep, and not immediately evident.
With both
meta-extract
andmeta-conduct
, if they're run with themetalad_studyminimeta
extractor on a dataset that doesn't have a.studyminimeta.yaml
file locally available, the error messages are not informative.Example 1:
Example 2:
Example 3: