search

snowflakedb / gosnowflake

Go Snowflake Driver
Apache License 2.0
292 stars 122 forks source link

SNOW-1646792: Support multiple result sets in a raw Arrow format #1205

Open CurtHagenlocher opened 2 weeks ago

CurtHagenlocher commented 2 weeks ago

What is the current behavior?

When using snowflakeConn.QueryArrowStream it's only possible to return a single result set.

What is the desired behavior?

Either via QueryArrowStream or some new function, it ought to be possible to use WithMultiStatement to execute multiple statements and get multiple results -- just as it is for the row-based interface.

How would this improve gosnowflake?

This is a generally useful scenario, and it would also go towards ultimately enabling https://github.com/apache/arrow-adbc/issues/1358.

sfc-gh-dszmolka commented 1 week ago

hi and thanks for submitting this issue. unfortunately, multi-statement query results returned as JSON are a limitation of the Snowflake backend, and thus affects all our drivers (not just gosnowflake). Same reason as https://github.com/snowflakedb/gosnowflake/issues/816. That said, it is entirely possible of course to submit a feature request for this, but it should be more like targeting the backend.

If you (or anyone reading this, needing the same feature) are already a Snowflake customer or partner, can you please reach out to your account team and let them know you would need Snowflake engine to support returning multi-statement query results as Arrow ?

I'm going to keep this issue open (indefinitely) for tracking, but please be aware that no progress can be made until backend is ready to fully support this. Thank you !