Closed schallerdavid closed 3 years ago
The composite tests fail with the following error:
Traceback (most recent call last): File "/home/david/github/experiments-binding-affinity/run_notebook.py", line 51, in <module> main() File "/home/david/github/experiments-binding-affinity/run_notebook.py", line 47, in main pm.execute_notebook(str(nbin), str(nbout), parameters) File "/home/david/miniconda3/envs/test_eba/lib/python3.9/site-packages/papermill/execute.py", line 118, in execute_notebook raise_for_execution_errors(nb, output_path) File "/home/david/miniconda3/envs/test_eba/lib/python3.9/site-packages/papermill/execute.py", line 230, in raise_for_execution_errors raise error papermill.exceptions.PapermillExecutionError: --------------------------------------------------------------------------- Exception encountered at "In [13]": --------------------------------------------------------------------------- ArrowNotImplementedError Traceback (most recent call last) /tmp/ipykernel_4232/462024357.py in <module> 8 path = OUT / f"{'__'.join([g for g in group if g != 'valid'])}.parquet" 9 parquets.append(path) ---> 10 ak.to_parquet(parquet, path) 11 # TODO: Missing indices? ~/miniconda3/envs/test_eba/lib/python3.9/site-packages/awkward/operations/convert.py in to_parquet(array, where, explode_records, list_to32, string_to32, bytestring_to32, **options) 3028 options["schema"] = first.schema 3029 -> 3030 writer = pyarrow.parquet.ParquetWriter(**options) 3031 writer.write_table(pyarrow.Table.from_batches([first])) 3032 ~/miniconda3/envs/test_eba/lib/python3.9/site-packages/pyarrow/parquet.py in __init__(self, where, schema, filesystem, flavor, version, use_dictionary, compression, write_statistics, use_deprecated_int96_timestamps, compression_level, use_byte_stream_split, writer_engine_version, data_page_version, use_compliant_nested_type, **options) 653 self._metadata_collector = options.pop('metadata_collector', None) 654 engine_version = 'V2' --> 655 self.writer = _parquet.ParquetWriter( 656 sink, schema, 657 version=version, ~/miniconda3/envs/test_eba/lib/python3.9/site-packages/pyarrow/_parquet.pyx in pyarrow._parquet.ParquetWriter.__cinit__() ~/miniconda3/envs/test_eba/lib/python3.9/site-packages/pyarrow/error.pxi in pyarrow.lib.check_status() ArrowNotImplementedError: Unhandled type for Arrow to Parquet schema conversion: dense_union<0: bool not null=0, 1: int64 not null=1>
The composite tests fail with the following error: