search

laughingman7743 / PyAthena

PyAthena is a Python DB API 2.0 (PEP 249) client for Amazon Athena.
MIT License
463 stars 105 forks source link

Add custom filesystem object to arrow engine #485

Closed jayceslesar closed 1 year ago

jayceslesar commented 1 year ago

pyarrow 13.0.0 doesnt work with AWS SSO but my org has a custom filesystem object that does, how could I go about injecting my custom s3 filesystem into pyathena instead of using the default one provided.

with engine.connect() as conn:
# How can I inject here?
jayceslesar commented 1 year ago

see https://github.com/apache/arrow/issues/38013

laughingman7743 commented 1 year ago

It cannot be changed to a custom S3 file system. https://github.com/laughingman7743/PyAthena/blob/master/pyathena/arrow/result_set.py#L86-L114