Missing support for PyArrow types in Pandas dataframes

aws / amazon-redshift-python-driver

Redshift Python Connector. It supports Python Database API Specification v2.0.

Apache License 2.0

204 stars 76 forks source link

Missing support for PyArrow types in Pandas dataframes #228

Open mattharrison opened 3 months ago

mattharrison commented 3 months ago

Most of the data ingestion utilities (pd.read_csv, pd.read_parquet, pd.read_sql) since Pandas 2.2 include a dtype_backend='pyarrow' option to use PyArrow types instead of NumPy backend.

It would be awesome if there was support for this.

Brooke-white commented 1 week ago

Hey @mattharrison , thank you for raising this issues and my apologies for the delay in response. I will discuss this request with the team and provide an update here.