Open sfc-gh-yuwang opened 1 month ago
Since this PR will not change any behavior of connector, but only speed up the to_pandas function, there is no test needed to be added. Instead, I ran a existed jenkins job to verify that this is a valid PR and does not break anything. Here is a link to successful run with snowpark: https://ci-dev-142.int.snowflakecomputing.com/job/SnowparkPythonSnowflakePythonClientRegressRunner/633/
Here is a link of research on to_pandas performance, which explains why this change works: https://docs.google.com/document/d/1HK7tNYoSLxQHSl7e_TkGxOZoz_Jqsjx0mhS6t6K_D_g/edit?usp=sharing
I have change the way this improvement is implemented, for now, the improvements only work when call to_pandas(), to_pandas_batches() and other function will not use this improvements
Please answer these questions before submitting your pull requests. Thanks!
What GitHub issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fill out the following pre-review checklist:
Please describe how your code solves the related issue.
Please write a short description of how your code change solves the related issue.