BigDaMa / abstraction-layer

Apache License 2.0
4 stars 8 forks source link

Exception if no error is detected -> no clean-up #6

Closed FelixNeutatz closed 5 years ago

FelixNeutatz commented 6 years ago

If the error detection method does not detect any error, e.g. katara_output.csv is empty, there will be an exception raised and no clean up is executed:

Traceback (most recent call last):
  File "cleaning_api.py", line 274, in <module>
    results_list = run_data_cleaning_job(run_input)
  File "cleaning_api.py", line 216, in run_data_cleaning_job
    return_list = run_katara(dataset_path, data_cleaning_job["tool"]["param"])
  File "cleaning_api.py", line 181, in run_katara
    detected_cells_list = read_csv_dataset(tool_results_path, header_exists=False)
  File "cleaning_api.py", line 62, in read_csv_dataset
    dataset_dataframe = pandas.read_csv(dataset_path, sep=",", header=None, encoding="utf-8", keep_default_na=False)
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 655, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 405, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 764, in __init__
    self._make_engine(self.engine)
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 985, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 1605, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 565, in pandas._libs.parsers.TextReader.__cinit__ (pandas/_libs/parsers.c:6260)
pandas.errors.EmptyDataError: No columns to parse from file
m-mahdavi commented 6 years ago

Yes. KATARA should not generate the output file when it cannot detect any error in the dataset. We need to fix that bug. Thanks.

m-mahdavi commented 5 years ago

Fixed.