coiled / dask-snowflake

Dask integration for Snowflake
BSD 3-Clause "New" or "Revised" License
30 stars 7 forks source link

``read_snowflake`` casts string columns to objects #63

Open phofl opened 3 months ago

phofl commented 3 months ago

the snowflake reader casts string columns in the to_pandas call to object, which is bad for memory, Gil contention and performance. We should utilise the same strategy as for read_parquet over in dask to preserve the string dtype when the option is set.