Open cpcloud opened 7 months ago
Describe the bug
A SQL query with coalesce fails when converted to pyarrow
coalesce
To Reproduce
In [1]: import datafusion as df In [2]: ctx = df.SessionContext() In [3]: ctx.sql Out[3]: <function SessionContext.sql(query)> In [4]: ctx.sql("select coalesce(null, 5)") Out[4]: DataFrame() +-------------------------+ | coalesce(NULL,Int64(5)) | +-------------------------+ | 5 | +-------------------------+ In [5]: ctx.sql("select coalesce(null, 5)").to_arrow_table() --------------------------------------------------------------------------- ArrowInvalid Traceback (most recent call last) Cell In[5], line 1 ----> 1 ctx.sql("select coalesce(null, 5)").to_arrow_table() File /nix/store/nqcbgqab0slp4kx3ixk8225nwrzy5mbd-python3-3.10.13-env/lib/python3.10/site-packages/pyarrow/table.pxi:4057, in pyarrow.lib.Table.from_batches() File /nix/store/nqcbgqab0slp4kx3ixk8225nwrzy5mbd-python3-3.10.13-env/lib/python3.10/site-packages/pyarrow/error.pxi:154, in pyarrow.lib.pyarrow_internal_check_status() File /nix/store/nqcbgqab0slp4kx3ixk8225nwrzy5mbd-python3-3.10.13-env/lib/python3.10/site-packages/pyarrow/error.pxi:91, in pyarrow.lib.check_status() ArrowInvalid: Schema at index 0 was different: coalesce(NULL,Int64(5)): int64 vs coalesce(NULL,Int64(5)): int64 not null
Expected behavior
I would expect the result to be a pyarrow table with 5 in it.
5
Describe the bug
A SQL query with
coalesce
fails when converted to pyarrowTo Reproduce
Expected behavior
I would expect the result to be a pyarrow table with
5
in it.