PrefectHQ / prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
https://prefect.io
Apache License 2.0
15.48k stars 1.52k forks source link

Add in fetch_pandas_all & fetch_pandas_batches from Snowflake connector #13086

Open sam-phinizy opened 1 year ago

sam-phinizy commented 1 year ago

Expectation / Proposal

Ideally it'd be nice to be able to use the fetch_pandas_* methods from the Snowflake connectors. Trying to lower the barrier to entry for data analysts/data scientists.

Traceback / Example

with SnowflakeConnector.load(config.snowflake_block_name, validate=False) as connector:
   data_frame = connect.pandas_fetch_all("SELECT * FROM COOL_TABLE")

print(df) # <- imagine a pandas printout here

I'm glad to implement this as I have time

ahuang11 commented 1 year ago

That's a good idea! One thought; should we name it fetch_pandas_all to reflect what's available in Snowflake? https://docs.snowflake.com/en/user-guide/python-connector-api#fetch_pandas_all