sfu-db / connector-x

Fastest library to load data from DB to DataFrames in Rust and Python
https://sfu-db.github.io/connector-x
MIT License
1.99k stars 161 forks source link

Oracle -> PanicException: Type LongRaw not implemented for oracle! #677

Open GuillaumePressiat opened 2 months ago

GuillaumePressiat commented 2 months ago

What language are you using?

Python

What version are you using?

connectorx-0.3.3-cp311-cp311-manylinux_2_28_x86_64.whl (60.5 MB)

What database are you using?

Oracle

What dataframe are you using?

polars

Can you describe your bug?

Panic exception when collecting an Oracle LONG RAW column datatype.

What are the steps to reproduce the behavior?

Table schema (data type LONG RAW)

CREATE TABLE SCHEM.TABL
(
  ID_FILE          NUMBER(15)   NOT NULL,
  FILE_CONTENT     LONG RAW
)
Example query / code
import polars as pl
sql_query = """
select file_content from schem.tabl
"""
df = pl.read_database_uri(sql_query, db_uri)

What is the error?

thread '<unnamed>' panicked at /__w/connector-x/connector-x/connectorx/src/sources/oracle/typesystem.rs:53:18:
not implemented: Type LongRaw not implemented for oracle!
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
---------------------------------------------------------------------------
PanicException                            Traceback (most recent call last)
File /opt/conda/lib/python3.11/site-packages/polars/io/database/_utils.py:54, in _read_sql_connectorx(query, connection_uri, partition_on, partition_range, partition_num, protocol, schema_overrides)
     53 try:
---> 54     tbl = cx.read_sql(
     55         conn=connection_uri,
     56         query=query,
     57         return_type="arrow2",
     58         partition_on=partition_on,
     59         partition_range=partition_range,
     60         partition_num=partition_num,
     61         protocol=protocol,
     62     )
     63 except BaseException as err:
     64     # basic sanitisation of /user:pass/ credentials exposed in connectorx errs

File /opt/conda/lib/python3.11/site-packages/connectorx/__init__.py:386, in read_sql(conn, query, return_type, protocol, partition_on, partition_range, partition_num, index_col)
    384 try_import_module("pyarrow")
--> 386 result = _read_sql(
    387     conn,
    388     "arrow2" if return_type in {"arrow2", "polars", "polars2"} else "arrow",
    389     queries=queries,
    390     protocol=protocol,
    391     partition_query=partition_query,
    392 )
    393 df = reconstruct_arrow(result)

PanicException: not implemented: Type LongRaw not implemented for oracle!

The above exception was the direct cause of the following exception:

PanicException                            Traceback (most recent call last)
Cell In[59], line 14
---> 14 u = pl.read_database_uri(sql_query, db_uri)

File /opt/conda/lib/python3.11/site-packages/polars/io/database/functions.py:395, in read_database_uri(query, uri, partition_on, partition_range, partition_num, protocol, engine, schema_overrides, execute_options)
    393         msg = "the 'connectorx' engine does not support use of `execute_options`"
    394         raise ValueError(msg)
--> 395     return _read_sql_connectorx(
    396         query,
    397         connection_uri=uri,
    398         partition_on=partition_on,
    399         partition_range=partition_range,
    400         partition_num=partition_num,
    401         protocol=protocol,
    402         schema_overrides=schema_overrides,
    403     )
    404 elif engine == "adbc":
    405     if not isinstance(query, str):

File /opt/conda/lib/python3.11/site-packages/polars/io/database/_utils.py:66, in _read_sql_connectorx(query, connection_uri, partition_on, partition_range, partition_num, protocol, schema_overrides)
     63 except BaseException as err:
     64     # basic sanitisation of /user:pass/ credentials exposed in connectorx errs
     65     errmsg = re.sub("://[^:]+:[^:]+@", "://***:***@", str(err))
---> 66     raise type(err)(errmsg) from err
     68 return from_arrow(tbl, schema_overrides=schema_overrides)

PanicException: not implemented: Type LongRaw not implemented for oracle!

My data are private so I don't put it here.

GuillaumePressiat commented 2 months ago

(with pandas read_sql the same query works).