Open richard-williamson opened 2 years ago
To clarify this request is different from https://github.com/GoogleCloudDataproc/spark-bigquery-connector/pull/305 which only pushes specified sql whereas Snowflake pushes spark logical queries back to RDBMS which abstracts need to use specialized syntax (and ideally allows PySpark dataframe join code to be pushed back as well)
Support similar optimizations that Snowflake supports: https://github.com/snowflakedb/spark-snowflake/pull/8/files