search

voltrondata-labs / benchmarks

Language-independent Continuous Benchmarking (CB) for Apache Arrow
MIT License
10 stars 11 forks source link

FileNotFoundError on wide-dataframe #147

Open pitrou opened 1 year ago

pitrou commented 1 year ago

It seems the wide-dataframe benchmark expects a benchmarks/data/temp directory that doesn't exist.

$ conbench wide-dataframe 
[...]
Traceback (most recent call last):
  [...]
  File "/home/antoine/arrow/benchmarks/benchmarks/wide_dataframe_benchmark.py", line 29, in run
    self._create_if_not_exists(path)
  File "/home/antoine/arrow/benchmarks/benchmarks/wide_dataframe_benchmark.py", line 46, in _create_if_not_exists
    parquet.write_table(table, path)
  File "/home/antoine/arrow/dev/python/pyarrow/parquet/core.py", line 3103, in write_table
    with ParquetWriter(
  File "/home/antoine/arrow/dev/python/pyarrow/parquet/core.py", line 1010, in __init__
    sink = self.file_handle = filesystem.open_output_stream(
  File "pyarrow/_fs.pyx", line 868, in pyarrow._fs.FileSystem.open_output_stream
  File "pyarrow/error.pxi", line 144, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 113, in pyarrow.lib.check_status
FileNotFoundError: [Errno 2] Failed to open local file '/home/antoine/arrow/benchmarks/benchmarks/data/temp/wide.parquet'. Detail: [errno 2] No such file or directory
alistaire47 commented 1 year ago

No such file or directory

Given it's trying to write, not read, I suspect this is a failure to recursively create directories