search

usc-isi-i2 / dsbox-ta2

The DSBox TA2 component
MIT License
11 stars 6 forks source link

Pickled pipeline failing #263

Closed kyao closed 4 years ago

kyao commented 5 years ago

Looks like many of the pipelines have been failing. We just have been catching the printing the exceptions. From the exception it almost look like that the pipeline edges are wrong. Primitives in the middle of the pipeline are Dataset, instead of Dataframe, as input.

Sample failure:

2019-06-11 18:35:12,235 [ERROR] dsbox.combinatorial_search.ConfigurationSpaceBaseSearch -- !!!! Test pickle failed
Traceback (most recent call last):
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 939, in _do_run_step
    self._run_step(step)
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 929, in _run_step
    self._run_primitive(step)
  File "/home/ktyao/dev/dsbox-2019-6-7/dsbox-ta2/python/dsbox/template/runtime.py", line 276, in _run_primitive
    super()._run_primitive(this_step)
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 849, in _run_primitive
    multi_call_result = self._call_primitive_method(primitive.multi_produce, multi_produce_arguments)
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 912, in _call_primitive_method
    raise error
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 908, in _call_primitive_method
    result = method(**arguments)
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/primitive_interfaces/base.py", line 437, in multi_produce
    return self._multi_produce(produce_methods=produce_methods, timeout=timeout, iterations=iterations, inputs=inputs)
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/primitive_interfaces/base.py", line 461, in _multi_produce
    results.append(getattr(self, method_name)(timeout=timeout, iterations=iterations, **arguments))
  File "/home/ktyao/dev/dsbox-2019-6-7/sklearn-wrap/sklearn_wrap/SKPCA.py", line 227, in produce
    sk_inputs = inputs.iloc[:, self._training_indices]
AttributeError: 'Dataset' object has no attribute 'iloc'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ktyao/dev/dsbox-2019-6-7/dsbox-ta2/python/dsbox/combinatorial_search/ConfigurationSpaceBaseSearch.py", line 527, in test_pickled_pipeline
    results = fitted_pipeline.produce(inputs=[test_dataset])
  File "/home/ktyao/dev/dsbox-2019-6-7/dsbox-ta2/python/dsbox/pipeline/fitted_pipeline.py", line 263, in produce
    self.runtime.produce(inputs, **arguments)
  File "/home/ktyao/dev/dsbox-2019-6-7/dsbox-ta2/python/dsbox/template/runtime.py", line 580, in produce
    self.check_results(self.produce_outputs)
  File "/home/ktyao/dev/dsbox-2019-6-7/dsbox-ta2/python/dsbox/template/runtime.py", line 587, in check_results
    raise res.error
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 970, in _run
    self._do_run()
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 956, in _do_run
    self._do_run_step(step)
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 948, in _do_run_step
    ) from error
d3m.exceptions.StepFailedError: Step 12 for pipeline a495b07f-7e63-4186-9ecb-39726c216bf4 failed.

Another failure:

2019-06-11 18:33:40,974 [ERROR] dsbox.combinatorial_search.ConfigurationSpaceBaseSearch -- !!!! Test pickle failed
Traceback (most recent call last):
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 939, in _do_run_step
    self._run_step(step)
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 929, in _run_step
    self._run_primitive(step)
  File "/home/ktyao/dev/dsbox-2019-6-7/dsbox-ta2/python/dsbox/template/runtime.py", line 276, in _run_primitive
    super()._run_primitive(this_step)
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 849, in _run_primitive
    multi_call_result = self._call_primitive_method(primitive.multi_produce, multi_produce_arguments)
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 912, in _call_primitive_method
    raise error
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 908, in _call_primitive_method
    result = method(**arguments)
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/primitive_interfaces/base.py", line 437, in multi_produce
    return self._multi_produce(produce_methods=produce_methods, timeout=timeout, iterations=iterations, inputs=inputs)
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/primitive_interfaces/base.py", line 461, in _multi_produce
    results.append(getattr(self, method_name)(timeout=timeout, iterations=iterations, **arguments))
  File "/home/ktyao/dev/dsbox-2019-6-7/common-primitives/common_primitives/extract_columns_semantic_types.py", line 82, in produce
    columns = self._get_columns(inputs.metadata, self.hyperparams)
  File "/home/ktyao/dev/dsbox-2019-6-7/common-primitives/common_primitives/extract_columns_semantic_types.py", line 97, in _get_columns
    semantic_types=hyperparams['semantic_types'],
ValueError: Input data has no columns matching semantic types: ('https://metadata.datadrivendiscovery.org/types/PrimaryKey', 'https://metadata.datadrivendiscovery.org/types/Attribute')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ktyao/dev/dsbox-2019-6-7/dsbox-ta2/python/dsbox/combinatorial_search/ConfigurationSpaceBaseSearch.py", line 527, in test_pickled_pipeline
    results = fitted_pipeline.produce(inputs=[test_dataset])
  File "/home/ktyao/dev/dsbox-2019-6-7/dsbox-ta2/python/dsbox/pipeline/fitted_pipeline.py", line 263, in produce
    self.runtime.produce(inputs, **arguments)
  File "/home/ktyao/dev/dsbox-2019-6-7/dsbox-ta2/python/dsbox/template/runtime.py", line 580, in produce
    self.check_results(self.produce_outputs)
  File "/home/ktyao/dev/dsbox-2019-6-7/dsbox-ta2/python/dsbox/template/runtime.py", line 587, in check_results
    raise res.error
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 970, in _run
    self._do_run()
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 956, in _do_run
    self._do_run_step(step)
  File "/home/ktyao/dev/dsbox-2019-6-7/d3m/d3m/runtime.py", line 948, in _do_run_step
    ) from error
d3m.exceptions.StepFailedError: Step 4 for pipeline 5209eb8c-2b87-4fc6-b626-a5e3ba8396a9 failed.