search

uncharted-distil / distil-auto-ml

Distil Automated Machine Learning Server
Apache License 2.0
2 stars 1 forks source link

Audio pipeline fails during test phase #44

Closed cdbethune closed 5 years ago

cdbethune commented 5 years ago

The following exception was generated:

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.6/dist-packages/d3m/runtime.py", line 2723, in <module>
    main(sys.argv)
  File "/usr/local/lib/python3.6/dist-packages/d3m/runtime.py", line 2719, in main
    handler(arguments, parser)
  File "/usr/local/lib/python3.6/dist-packages/d3m/runtime.py", line 2287, in handler
    arguments.runtime_handler(arguments, pipeline_resolver=pipeline_resolver, meta_parser=meta_parser, dataset_resolver=dataset_resolver, problem_resolver=problem_resolver)
  File "/usr/local/lib/python3.6/dist-packages/d3m/runtime.py", line 2022, in _fit_score
    runtime_environment=runtime_environment,
  File "/usr/local/lib/python3.6/dist-packages/d3m/runtime.py", line 1126, in fit
    result = runtime.fit(inputs, return_values=['outputs.0'])
  File "/usr/local/lib/python3.6/dist-packages/d3m/runtime.py", line 1003, in fit
    return self._run(inputs, metadata_base.PipelineRunPhase.FIT, return_values)
  File "/usr/local/lib/python3.6/dist-packages/d3m/runtime.py", line 935, in _run
    self._previous_pipeline_run_id = pipeline_run.get_id()
  File "/usr/local/lib/python3.6/dist-packages/d3m/metadata/pipeline_run.py", line 668, in get_id
    return self._to_json_structure()['id']
  File "/usr/local/lib/python3.6/dist-packages/d3m/metadata/pipeline_run.py", line 363, in _to_json_structure
    json_structure['id'] = utils.compute_hash_id(json_structure)
  File "/usr/local/lib/python3.6/dist-packages/d3m/utils.py", line 867, in compute_hash_id
    return str(uuid.uuid5(HASH_ID_NAMESPACE, json.dumps(obj, sort_keys=True)))
  File "/usr/lib/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'DataFrame' is not JSON serializable
ta3-uncharted@k8s-jump-server:~/may2019$ ls
README.md  distil-auto-ml.yaml
ta3-uncharted@k8s-jump-server:~/may2019$ vi ~/output/
logs/                 pipelines_ranked/     predictions/          scoring_pipeline.yml
pipeline_runs/        pipelines_scored/     score/                supporting_files/
ta3-uncharted@k8s-jump-server:~/may2019$ vi ~/output/predictions/73ae8493-f856-449f-ac37-707e5c013e4d
73ae8493-f856-449f-ac37-707e5c013e4d/     73ae8493-f856-449f-ac37-707e5c013e4d.csv
ta3-uncharted@k8s-jump-server:~/may2019$ vi ~/output/predictions/73ae8493-f856-449f-ac37-707e5c013e4d
73ae8493-f856-449f-ac37-707e5c013e4d/     73ae8493-f856-449f-ac37-707e5c013e4d.csv
ta3-uncharted@k8s-jump-server:~/may2019$ vi ~/output/predictions/73ae8493-f856-449f-ac37-707e5c013e4d.csv
jbartonthomas commented 5 years ago

Right now, exline.primitives.audio_loader.py passes the large audio dataset as a standard dataframe:

image

I am wondering if this is causing issues with pipeline serialization.