search

snowflakedb / snowflake-connector-python

Snowflake Connector for Python
https://pypi.python.org/pypi/snowflake-connector-python/
Apache License 2.0
568 stars 458 forks source link

SNOW-1366543: Log Error coming from line CArrowTableIterator.cpp#L230 making a lot noise #1939

Closed rajathpatel23 closed 1 month ago

rajathpatel23 commented 2 months ago

We are getting this log error while processing data from Snowflake. It seems to be making a lot of noise; I would like to understand - what was the change made in the snowflake ticket? I feel the logger there is more of debug logger than actual log error.

The changes introduced in this PR seem to be start to give us the log error

Could you help me with this? Also, can you provide more info on the support for pandas that have come with this change?

rajathpatel23 commented 2 months ago

I tried raising a support ticket - however I could not create one, hence having this issue as a placeholder for now.

sfc-gh-jrose commented 2 months ago

Hey Rajat, That error log is indeed very noisy. It is left over from a debug session that I had while working on the PR you mentioned and does not have a lot of value outside of that. I've created a PR here to remove it: #1942

That PR was for adding support for structured types to the pandas_fetch_all function. Currently most nested data returned by the connector comes as json encoded strings, but in the future it will support returning structured data instead. This was one part of the overall change to add that support.

sfc-gh-dszmolka commented 2 months ago

hi - thank you for submitting this issue with us.

Can you please elaborate what kind of log messages exactly you mean? Do you have a snippet from a log where the unexpected loglines would be visible ? Please make sure to sanitize them from any account etc. details.

We have some more information about the structured data types mentioned in the fetch_pandas_all changelog entry, please take a peek here if you're interested in this Public Preview feature.

sfc-gh-dszmolka commented 2 months ago

ah, late to the party :)

rajathpatel23 commented 2 months ago

@sfc-gh-jrose Thank you so much for taking a look at this quickly.

parinpshah94 commented 1 month ago

@sfc-gh-jrose Thanks for making this fix, do you know when a release with this fix will be published to PyPi?

sfc-gh-jrose commented 1 month ago

The version for this fix should be published later this week.

sfc-gh-dszmolka commented 1 month ago

version 3.10.1 released with the fix