search

snowflakedb / snowflake-connector-python

Snowflake Connector for Python
https://pypi.python.org/pypi/snowflake-connector-python/
Apache License 2.0
601 stars 473 forks source link

SNOW-884196: Snowflake connector write_pandas inserts datetime64[ns] to Snowflake as an Invalid Date #1677

Closed dwelden closed 1 year ago

dwelden commented 1 year ago

Python version

Python 3.10.5

Operating system and processor architecture

Windows-10-10.0.19044-SP0

Installed packages

argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asn1crypto==1.5.1
asttokens==2.0.8
attrs==22.1.0
backcall==0.2.0
beautifulsoup4==4.11.1
bleach==5.0.1
certifi==2022.6.15
cffi==1.15.1
charset-normalizer==2.1.1
cloudpickle==2.0.0
colorama==0.4.5
cryptography==36.0.2
debugpy==1.6.3
decorator==5.1.1
defusedxml==0.7.1
entrypoints==0.4
executing==1.0.0
fastjsonschema==2.16.1
filelock==3.8.0
idna==3.3
ipykernel==6.15.2
ipython==8.5.0
ipython-genutils==0.2.0
jedi==0.18.1
Jinja2==3.1.2
jsonschema==4.15.0
jupyter-core==4.11.1
jupyter_client==7.3.5
jupyterlab-pygments==0.2.2
lxml==4.9.1
MarkupSafe==2.1.1
matplotlib-inline==0.1.6
mistune==2.0.4
nbclient==0.6.7
nbconvert==7.0.0
nbformat==5.4.0
nest-asyncio==1.5.5
notebook==6.4.12
numpy==1.25.1
oscrypto==1.3.0
packaging==21.3
pandas==1.4.4
pandocfilters==1.5.0
parso==0.8.3
pickleshare==0.7.5
Pillow==9.5.0
prettytable==3.5.0
prometheus-client==0.14.1
prompt-toolkit==3.0.31
psutil==5.9.2
pure-eval==0.2.2
pyarrow==8.0.0
pycparser==2.21
pycryptodomex==3.15.0
pydenticon==0.3.1
Pygments==2.13.0
PyJWT==2.4.0
pyOpenSSL==22.0.0
pyparsing==3.0.9
pyrsistent==0.18.1
PySimpleGUI==4.60.3
python-dateutil==2.8.2
python-ulid==1.1.0
pytz==2022.2.1
pywin32==304
pywinpty==2.0.7
pyzmq==23.2.1
requests==2.28.1
Send2Trash==1.8.0
six==1.16.0
snowflake-connector-python==2.7.12
snowflake-snowpark-python==1.5.1
soupsieve==2.3.2.post1
stack-data==0.5.0
terminado==0.15.0
tinycss2==1.1.1
tornado==6.2
traitlets==5.3.0
typing_extensions==4.3.0
urllib3==1.26.12
wcwidth==0.2.5
webencodings==0.5.1

What did you do?

import pandas as pd
from snowflake.connector.pandas_tools import write_pandas

df = pd.read_sas(source_file, format='sas7bdat', encoding='cp1252')
success, _, nRows, output = write_pandas(cnxn, df, target_table, overwrite=overwrite)

>>> df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 152130 entries, 0 to 152129
Data columns (total 5 columns):
 #   Column              Non-Null Count   Dtype
---  ------              --------------   -----
 0   ProductKey          152130 non-null  object
 1   DistributionCenter  152130 non-null  object
 2   DATE                152130 non-null  datetime64[ns]
 3   Discount            152130 non-null  float64
 4   Revenue             151830 non-null  float64
dtypes: datetime64[ns](1), float64(2), object(2)
memory usage: 5.8+ MB

What did you expect to see?

Expected Pandas datetime column to be written to Snowflake as a TIMESTAMP_NTZ. Dates are not loading properly.

PRODUCTKEY DISTRIBUTIONCENTER DATE DISCOUNT REVENUE
Cleanser Cary Invalid Date 0 0
Shampoo Cary Invalid Date 0 0
... ... ... ... ...

Can you set logging to DEBUG and collect the logs?

2023-08-02 14:41:27,622 - MainThread connection.py:270 - __init__() - INFO - Snowflake Connector for Python Version: 2.7.12, Python Version: 3.10.5, Platform: Windows-10-10.0.19044-SP0
2023-08-02 14:41:27,622 - MainThread connection.py:512 - connect() - DEBUG - connect
2023-08-02 14:41:27,623 - MainThread connection.py:802 - __config() - DEBUG - __config
2023-08-02 14:41:27,623 - MainThread connection.py:926 - __config() - INFO - This connection is in OCSP Fail Open Mode. TLS Certificates would be checked for validity and revocation status. Any other Certificate Revocation related exceptions or OCSP Responder failures would be disregarded in favor of connectivity.
2023-08-02 14:41:27,623 - MainThread connection.py:944 - __config() - INFO - Setting use_openssl_only mode to False
2023-08-02 14:41:27,624 - MainThread converter.py:145 - __init__() - DEBUG - use_numpy: False
2023-08-02 14:41:27,624 - MainThread converter_issue23517.py:27 - __init__() - DEBUG - initialized
2023-08-02 14:41:27,624 - MainThread connection.py:705 - __open_connection() - DEBUG - REST API object was created: myaccount.snowflakecomputing.com:443
2023-08-02 14:41:27,624 - MainThread auth.py:460 - _read_temporary_credential() - DEBUG - Dependency 'keyring' is not installed, cannot cache id token. You might experience multiple authentication pop ups while using ExternalBrowser Authenticator. To avoid this please install keyring module using the following command : pip install snowflake-connector-python[secure-local-storage]
2023-08-02 14:41:27,624 - MainThread auth_webbrowser.py:92 - authenticate() - DEBUG - authenticating by Web Browser
2023-08-02 14:41:27,635 - MainThread auth_webbrowser.py:126 - authenticate() - DEBUG - step 1: query GS to obtain SSO url
2023-08-02 14:41:27,636 - MainThread auth_webbrowser.py:354 - _get_sso_url() - DEBUG - account=myaccount, authenticator=EXTERNALBROWSER, user=Dave.Welden
2023-08-02 14:41:27,636 - MainThread retry.py:351 - from_int() - DEBUG - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
2023-08-02 14:41:27,637 - MainThread retry.py:351 - from_int() - DEBUG - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
2023-08-02 14:41:27,637 - MainThread network.py:1147 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-08-02 14:41:27,637 - MainThread network.py:827 - _request_exec_wrapper() - DEBUG - remaining request timeout: 120, retry cnt: 1
2023-08-02 14:41:27,637 - MainThread network.py:808 - add_request_guid() - DEBUG - Request guid: 31674d7d-d43b-444e-aaf0-4ffc74fa4278
2023-08-02 14:41:27,637 - MainThread network.py:1006 - _request_exec() - DEBUG - socket timeout: 60
2023-08-02 14:41:27,640 - MainThread connectionpool.py:1003 - _new_conn() - DEBUG - Starting new HTTPS connection (1): myaccount.snowflakecomputing.com:443
2023-08-02 14:41:28,401 - MainThread cache.py:490 - _save() - DEBUG - acquiring C:\Users\weldend\AppData\Local\Snowflake\Caches\ocsp_cache.lock timed out, skipping saving...
2023-08-02 14:41:28,409 - MainThread ocsp_snowflake.py:507 - reset_cache_dir() - DEBUG - cache directory: C:\Users\weldend\AppData\Local\Snowflake\Caches
2023-08-02 14:41:28,493 - MainThread ssl_wrap_socket.py:80 - ssl_wrap_socket_with_ocsp() - DEBUG - OCSP Mode: FAIL_OPEN, OCSP response cache file name: None
2023-08-02 14:41:28,493 - MainThread ocsp_snowflake.py:549 - reset_ocsp_response_cache_uri() - DEBUG - ocsp_response_cache_uri: file://C:/Users/weldend/AppData/Local/Snowflake/Caches/ocsp_response_cache.json
2023-08-02 14:41:28,494 - MainThread ocsp_snowflake.py:550 - reset_ocsp_response_cache_uri() - DEBUG - OCSP_VALIDATION_CACHE size: 0
2023-08-02 14:41:28,494 - MainThread ocsp_snowflake.py:332 - reset_ocsp_dynamic_cache_server_url() - DEBUG - OCSP response cache server is enabled: http://ocsp.snowflakecomputing.com/ocsp_response_cache.json
2023-08-02 14:41:28,494 - MainThread ocsp_snowflake.py:368 - reset_ocsp_dynamic_cache_server_url() - DEBUG - OCSP dynamic cache server RETRY URL: None
2023-08-02 14:41:28,500 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:45:34+00:00
2023-08-02 14:41:28,503 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-10-15 00:00:00+00:00
2023-08-02 14:41:28,505 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-05-12 23:58:59+00:00
2023-08-02 14:41:28,508 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-05-12 23:58:59+00:00
2023-08-02 14:41:28,511 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:41:06+00:00
2023-08-02 14:41:28,513 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:41:06+00:00
2023-08-02 14:41:28,522 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:28,543 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:28,544 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:28,546 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:28,548 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:28,550 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:28,552 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:28,554 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:28,558 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:28,559 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:28,561 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:28,563 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:28,565 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:28,566 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:28,568 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:28,570 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:28,572 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:28,574 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:28,576 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:28,578 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:28,598 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-08-28 15:50:11+00:00
2023-08-02 14:41:28,601 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-08-28 15:50:08+00:00
2023-08-02 14:41:28,604 - MainThread ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-08-28 15:50:11+00:00
2023-08-02 14:41:28,611 - MainThread ocsp_snowflake.py:585 - read_ocsp_response_cache_file() - DEBUG - Read OCSP response cache file: C:/Users/weldend/AppData/Local/Snowflake/Caches/ocsp_response_cache.json, count=188
2023-08-02 14:41:28,612 - MainThread ocsp_snowflake.py:1118 - validate() - DEBUG - validating certificate: myaccount.snowflakecomputing.com
2023-08-02 14:41:28,612 - MainThread ocsp_asn1crypto.py:439 - extract_certificate_chain() - DEBUG - # of certificates: 2
2023-08-02 14:41:28,613 - MainThread ocsp_asn1crypto.py:444 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'California'), ('locality_name', 'San Mateo'), ('organization_name', 'Snowflake Inc.'), ('common_name', '*.east-us-2.azure.snowflakecomputing.com')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('common_name', 'DigiCert Global G2 TLS RSA SHA256 2020 CA1')])
2023-08-02 14:41:28,613 - MainThread ocsp_asn1crypto.py:444 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('common_name', 'DigiCert Global G2 TLS RSA SHA256 2020 CA1')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('organizational_unit_name', 'www.digicert.com'), ('common_name', 'DigiCert Global Root G2')])
2023-08-02 14:41:28,615 - MainThread ocsp_asn1crypto.py:123 - read_cert_bundle() - DEBUG - reading certificate bundle: C:\certs\cacert.pem
2023-08-02 14:41:28,636 - MainThread ocsp_asn1crypto.py:465 - create_pair_issuer_subject() - DEBUG - not found issuer_der: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('organizational_unit_name', 'www.digicert.com'), ('common_name', 'DigiCert Global Root G2')])
2023-08-02 14:41:28,638 - MainThread ocsp_snowflake.py:725 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'California'), ('locality_name', 'San Mateo'), ('organization_name', 'Snowflake Inc.'), ('common_name', '*.east-us-2.azure.snowflakecomputing.com')])
2023-08-02 14:41:28,640 - MainThread ocsp_snowflake.py:725 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('common_name', 'DigiCert Global G2 TLS RSA SHA256 2020 CA1')])
2023-08-02 14:41:28,642 - MainThread ocsp_snowflake.py:647 - write_ocsp_response_cache_file() - DEBUG - writing OCSP response cache file to C:/Users/weldend/AppData/Local/Snowflake/Caches/ocsp_response_cache.json
2023-08-02 14:41:28,643 - MainThread ocsp_snowflake.py:1794 - encode_ocsp_response_cache() - DEBUG - encoding OCSP response cache to JSON
2023-08-02 14:41:28,685 - MainThread ocsp_snowflake.py:1175 - _validate() - DEBUG - ok
2023-08-02 14:41:28,865 - MainThread connectionpool.py:456 - _make_request() - DEBUG - https://myaccount.snowflakecomputing.com:443 "POST /session/authenticator-request?request_guid=31674d7d-d43b-444e-aaf0-4ffc74fa4278 HTTP/1.1" 200 905
2023-08-02 14:41:28,866 - MainThread network.py:1032 - _request_exec() - DEBUG - SUCCESS
2023-08-02 14:41:28,866 - MainThread network.py:1152 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-08-02 14:41:28,866 - MainThread network.py:715 - _post_request() - DEBUG - ret[code] = None, after post request
2023-08-02 14:41:28,866 - MainThread auth_webbrowser.py:131 - authenticate() - DEBUG - step 2: open a browser
2023-08-02 14:41:29,678 - MainThread auth_webbrowser.py:155 - authenticate() - DEBUG - step 3: accept SAML token
2023-08-02 14:41:42,367 - MainThread auth_webbrowser.py:325 - _get_user_agent() - DEBUG - User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.188
2023-08-02 14:41:42,368 - MainThread auth.py:170 - authenticate() - DEBUG - authenticate
2023-08-02 14:41:42,368 - MainThread auth.py:200 - authenticate() - DEBUG - assertion content: biglongtoken==
2023-08-02 14:41:42,369 - MainThread auth.py:203 - authenticate() - DEBUG - account=myaccount, user=Dave.Welden, database=WELDEND, schema=SSDP, warehouse=MYWH, role=MYROLE, request_id=c2dec081-01d3-4797-9e3e-19dd473c04a0
2023-08-02 14:41:42,369 - MainThread auth.py:236 - authenticate() - DEBUG - body['data']: {'CLIENT_APP_ID': 'PythonConnector', 'CLIENT_APP_VERSION': '2.7.12', 'SVN_REVISION': None, 'ACCOUNT_NAME': 'myaccount', 'LOGIN_NAME': 'Dave.Welden', 'CLIENT_ENVIRONMENT': {'APPLICATION': 'PythonConnector', 'OS': 'Windows', 'OS_VERSION': 'Windows-10-10.0.19044-SP0', 'PYTHON_VERSION': '3.10.5', 'PYTHON_RUNTIME': 'CPython', 'PYTHON_COMPILER': 'MSC v.1929 64 bit (AMD64)', 'OCSP_MODE': 'FAIL_OPEN', 'TRACING': 10, 'LOGIN_TIMEOUT': 120, 'NETWORK_TIMEOUT': None}, 'AUTHENTICATOR': 'EXTERNALBROWSER', 'TOKEN': biglongtoken==', 'PROOF_KEY': 'proofkey=_-1_0', 'SESSION_PARAMETERS': {'CLIENT_PREFETCH_THREADS': 4, 'CLIENT_STORE_TEMPORARY_CREDENTIAL': True}}
2023-08-02 14:41:42,370 - MainThread auth.py:254 - authenticate() - DEBUG - Timeout set to 120
2023-08-02 14:41:42,370 - MainThread network.py:1147 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-08-02 14:41:42,370 - MainThread network.py:827 - _request_exec_wrapper() - DEBUG - remaining request timeout: 120, retry cnt: 1
2023-08-02 14:41:42,371 - MainThread network.py:808 - add_request_guid() - DEBUG - Request guid: ebb0c0e9-285c-4d87-8649-47eb044c41ac
2023-08-02 14:41:42,371 - MainThread network.py:1006 - _request_exec() - DEBUG - socket timeout: 60
2023-08-02 14:41:42,750 - MainThread connectionpool.py:456 - _make_request() - DEBUG - https://myaccount.snowflakecomputing.com:443 "POST /session/v1/login-request?request_id=c2dec081-01d3-4797-9e3e-19dd473c04a0&databaseName=WELDEND&schemaName=SSDP&warehouse=MYWH&roleName=MYROLE&request_guid=ebb0c0e9-285c-4d87-8649-47eb044c41ac HTTP/1.1" 200 None
2023-08-02 14:41:42,752 - MainThread network.py:1032 - _request_exec() - DEBUG - SUCCESS
2023-08-02 14:41:42,752 - MainThread network.py:1152 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-08-02 14:41:42,753 - MainThread network.py:715 - _post_request() - DEBUG - ret[code] = None, after post request
2023-08-02 14:41:42,753 - MainThread auth.py:373 - authenticate() - DEBUG - completed authentication
2023-08-02 14:41:42,754 - MainThread auth.py:416 - authenticate() - DEBUG - token = ******
2023-08-02 14:41:42,754 - MainThread auth.py:419 - authenticate() - DEBUG - master_token = ******
2023-08-02 14:41:42,754 - MainThread auth.py:423 - authenticate() - DEBUG - id_token = ******
2023-08-02 14:41:42,755 - MainThread auth.py:427 - authenticate() - DEBUG - mfa_token = NULL
2023-08-02 14:41:42,755 - MainThread auth.py:503 - _write_temporary_credential() - DEBUG - Dependency 'keyring' is not installed, cannot cache id token. You might experience multiple authentication pop ups while using ExternalBrowser Authenticator. To avoid this please install keyring module using the following command : pip install snowflake-connector-python[secure-local-storage]
2023-08-02 14:41:42,756 - MainThread connection.py:624 - cursor() - DEBUG - cursor
2023-08-02 14:41:42,756 - MainThread pandas_tools.py:174 - write_pandas() - DEBUG - creating stage with 'create temporary stage /* Python:snowflake.connector.pandas_tools.write_pandas() */ "cbpzl"'
2023-08-02 14:41:42,757 - MainThread cursor.py:630 - execute() - DEBUG - executing SQL/command
2023-08-02 14:41:42,758 - MainThread cursor.py:669 - execute() - DEBUG - binding: [create temporary stage /* Python:snowflake.connector.pandas_tools.write_pandas()...] with input=[None], processed=[{}]
2023-08-02 14:41:42,758 - MainThread cursor.py:715 - execute() - INFO - query: [create temporary stage /* Python:snowflake.connector.pandas_tools.write_pandas()...]
2023-08-02 14:41:42,758 - MainThread connection.py:1302 - _next_sequence_counter() - DEBUG - sequence counter: 1
2023-08-02 14:41:42,758 - MainThread cursor.py:460 - _execute_helper() - DEBUG - Request id: addaef02-bf48-44ee-86f9-77068f3cb80d
2023-08-02 14:41:42,758 - MainThread cursor.py:462 - _execute_helper() - DEBUG - running query [create temporary stage /* Python:snowflake.connector.pandas_tools.write_pandas()...]
2023-08-02 14:41:42,759 - MainThread cursor.py:471 - _execute_helper() - DEBUG - is_file_transfer: False
2023-08-02 14:41:42,760 - MainThread connection.py:972 - cmd_query() - DEBUG - _cmd_query
2023-08-02 14:41:42,760 - MainThread connection.py:995 - cmd_query() - DEBUG - sql=[create temporary stage /* Python:snowflake.connector.pandas_tools.write_pandas()...], sequence_id=[1], is_file_transfer=[False]
2023-08-02 14:41:42,760 - MainThread network.py:1147 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-08-02 14:41:42,761 - MainThread network.py:827 - _request_exec_wrapper() - DEBUG - remaining request timeout: None, retry cnt: 1
2023-08-02 14:41:42,762 - MainThread network.py:808 - add_request_guid() - DEBUG - Request guid: 40bfa6ff-78ea-4536-874e-c0b3ce53a4be
2023-08-02 14:41:42,763 - MainThread network.py:1006 - _request_exec() - DEBUG - socket timeout: 60
2023-08-02 14:41:43,032 - MainThread connectionpool.py:456 - _make_request() - DEBUG - https://myaccount.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=addaef02-bf48-44ee-86f9-77068f3cb80d&request_guid=40bfa6ff-78ea-4536-874e-c0b3ce53a4be HTTP/1.1" 200 None
2023-08-02 14:41:43,047 - MainThread network.py:1032 - _request_exec() - DEBUG - SUCCESS
2023-08-02 14:41:43,048 - MainThread network.py:1152 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-08-02 14:41:43,049 - MainThread network.py:715 - _post_request() - DEBUG - ret[code] = None, after post request
2023-08-02 14:41:43,049 - MainThread network.py:739 - _post_request() - DEBUG - Query id: 01ae0b7d-0c04-d5f7-0000-09cd28349eb6
2023-08-02 14:41:43,050 - MainThread cursor.py:737 - execute() - DEBUG - sfqid: 01ae0b7d-0c04-d5f7-0000-09cd28349eb6
2023-08-02 14:41:43,050 - MainThread cursor.py:739 - execute() - INFO - query execution done
2023-08-02 14:41:43,051 - MainThread cursor.py:741 - execute() - DEBUG - SUCCESS
2023-08-02 14:41:43,051 - MainThread cursor.py:744 - execute() - DEBUG - PUT OR GET: None
2023-08-02 14:41:43,051 - MainThread cursor.py:841 - _init_result_and_meta() - DEBUG - Query result format: json
2023-08-02 14:41:43,051 - MainThread result_batch.py:440 - _parse() - DEBUG - parsing for result batch id: 1
2023-08-02 14:41:43,052 - MainThread result_set.py:58 - result_set_iterator() - DEBUG - beginning to schedule result batch downloads
2023-08-02 14:41:43,642 - MainThread pandas_tools.py:196 - write_pandas() - DEBUG - uploading files with 'PUT /* Python:snowflake.connector.pandas_tools.write_pandas() */ 'file://C:\\Users\\weldend\\AppData\\Local\\Temp\\tmpyrm1wpy0\\file0.txt' @"cbpzl" PARALLEL=4'
2023-08-02 14:41:43,643 - MainThread cursor.py:630 - execute() - DEBUG - executing SQL/command
2023-08-02 14:41:43,644 - MainThread cursor.py:669 - execute() - DEBUG - binding: [PUT /* Python:snowflake.connector.pandas_tools.write_pandas() */ 'file://C:\\Use...] with input=[None], processed=[{}]
2023-08-02 14:41:43,644 - MainThread cursor.py:715 - execute() - INFO - query: [PUT /* Python:snowflake.connector.pandas_tools.write_pandas() */ 'file://C:\\Use...]
2023-08-02 14:41:43,645 - MainThread connection.py:1302 - _next_sequence_counter() - DEBUG - sequence counter: 2
2023-08-02 14:41:43,645 - MainThread cursor.py:460 - _execute_helper() - DEBUG - Request id: cfe86520-5473-400f-9a6c-74a53128eebb
2023-08-02 14:41:43,645 - MainThread cursor.py:462 - _execute_helper() - DEBUG - running query [PUT /* Python:snowflake.connector.pandas_tools.write_pandas() */ 'file://C:\\Use...]
2023-08-02 14:41:43,646 - MainThread cursor.py:471 - _execute_helper() - DEBUG - is_file_transfer: True
2023-08-02 14:41:43,646 - MainThread connection.py:972 - cmd_query() - DEBUG - _cmd_query
2023-08-02 14:41:43,647 - MainThread connection.py:995 - cmd_query() - DEBUG - sql=[PUT /* Python:snowflake.connector.pandas_tools.write_pandas() */ 'file://C:\\Use...], sequence_id=[2], is_file_transfer=[True]
2023-08-02 14:41:43,648 - MainThread network.py:1147 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-08-02 14:41:43,648 - MainThread network.py:827 - _request_exec_wrapper() - DEBUG - remaining request timeout: None, retry cnt: 1
2023-08-02 14:41:43,649 - MainThread network.py:808 - add_request_guid() - DEBUG - Request guid: 9614a1e0-b065-4566-b244-7c461e16e7e5
2023-08-02 14:41:43,649 - MainThread network.py:1006 - _request_exec() - DEBUG - socket timeout: 60
2023-08-02 14:41:43,865 - MainThread connectionpool.py:456 - _make_request() - DEBUG - https://myaccount.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=cfe86520-5473-400f-9a6c-74a53128eebb&request_guid=9614a1e0-b065-4566-b244-7c461e16e7e5 HTTP/1.1" 200 None
2023-08-02 14:41:43,867 - MainThread network.py:1032 - _request_exec() - DEBUG - SUCCESS
2023-08-02 14:41:43,868 - MainThread network.py:1152 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-08-02 14:41:43,868 - MainThread network.py:715 - _post_request() - DEBUG - ret[code] = None, after post request
2023-08-02 14:41:43,868 - MainThread network.py:739 - _post_request() - DEBUG - Query id: 01ae0b7d-0c04-d5f7-0000-09cd28349ebe
2023-08-02 14:41:43,869 - MainThread cursor.py:737 - execute() - DEBUG - sfqid: 01ae0b7d-0c04-d5f7-0000-09cd28349ebe
2023-08-02 14:41:43,869 - MainThread cursor.py:739 - execute() - INFO - query execution done
2023-08-02 14:41:43,869 - MainThread cursor.py:741 - execute() - DEBUG - SUCCESS
2023-08-02 14:41:43,869 - MainThread cursor.py:744 - execute() - DEBUG - PUT OR GET: <re.Match object; span=(0, 4), match='PUT '>
2023-08-02 14:41:43,870 - MainThread file_transfer_agent.py:822 - _init_encryption_material() - DEBUG - UPLOAD
2023-08-02 14:41:43,871 - MainThread file_transfer_agent.py:949 - _init_file_metadata() - DEBUG - command type: UPLOAD
2023-08-02 14:41:44,206 - MainThread file_transfer_agent.py:1096 - _process_file_compression_type() - DEBUG - detected the encoding parquet: file=C:\Users\weldend\AppData\Local\Temp\tmpyrm1wpy0\file0.txt
2023-08-02 14:41:44,207 - MainThread file_transfer_agent.py:392 - execute() - DEBUG - parallel=[4]
2023-08-02 14:41:44,207 - MainThread file_transfer_agent.py:415 - transfer() - DEBUG - Chunk ThreadPoolExecutor size: 4
2023-08-02 14:41:44,213 - ThreadPoolExecutor-3_0 storage_client.py:189 - preprocess() - DEBUG - Preprocessing C:\Users\weldend\AppData\Local\Temp\tmpyrm1wpy0\file0.txt
2023-08-02 14:41:44,214 - ThreadPoolExecutor-3_0 retry.py:351 - from_int() - DEBUG - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
2023-08-02 14:41:44,215 - ThreadPoolExecutor-3_0 retry.py:351 - from_int() - DEBUG - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
2023-08-02 14:41:44,215 - ThreadPoolExecutor-3_0 network.py:1147 - _use_requests_session() - DEBUG - Session status for SessionPool 'someblob.blob.core.windows.net', SessionPool 1/1 active sessions
2023-08-02 14:41:44,215 - ThreadPoolExecutor-3_0 storage_client.py:277 - _send_request_with_retry() - DEBUG - storage client request with session <snowflake.connector.vendored.requests.sessions.Session object at 0x00000209342FD5A0>
2023-08-02 14:41:44,226 - ThreadPoolExecutor-3_0 connectionpool.py:1003 - _new_conn() - DEBUG - Starting new HTTPS connection (1): someblob.blob.core.windows.net:443
2023-08-02 14:41:44,673 - ThreadPoolExecutor-3_0 ssl_wrap_socket.py:80 - ssl_wrap_socket_with_ocsp() - DEBUG - OCSP Mode: FAIL_OPEN, OCSP response cache file name: None
2023-08-02 14:41:44,673 - ThreadPoolExecutor-3_0 ocsp_snowflake.py:549 - reset_ocsp_response_cache_uri() - DEBUG - ocsp_response_cache_uri: file://C:/Users/weldend/AppData/Local/Snowflake/Caches/ocsp_response_cache.json
2023-08-02 14:41:44,674 - ThreadPoolExecutor-3_0 ocsp_snowflake.py:550 - reset_ocsp_response_cache_uri() - DEBUG - OCSP_VALIDATION_CACHE size: 188
2023-08-02 14:41:44,674 - ThreadPoolExecutor-3_0 ocsp_snowflake.py:332 - reset_ocsp_dynamic_cache_server_url() - DEBUG - OCSP response cache server is enabled: http://ocsp.snowflakecomputing.com/ocsp_response_cache.json
2023-08-02 14:41:44,674 - ThreadPoolExecutor-3_0 ocsp_snowflake.py:368 - reset_ocsp_dynamic_cache_server_url() - DEBUG - OCSP dynamic cache server RETRY URL: None
2023-08-02 14:41:44,689 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:45:34+00:00
2023-08-02 14:41:44,694 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:45:34+00:00
2023-08-02 14:41:44,705 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-10-15 00:00:00+00:00
2023-08-02 14:41:44,711 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-10-15 00:00:00+00:00
2023-08-02 14:41:44,717 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-05-12 23:58:59+00:00
2023-08-02 14:41:44,723 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-05-12 23:58:59+00:00
2023-08-02 14:41:44,728 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-05-12 23:58:59+00:00
2023-08-02 14:41:44,734 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-05-12 23:58:59+00:00
2023-08-02 14:41:44,746 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:41:06+00:00
2023-08-02 14:41:44,752 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:41:06+00:00
2023-08-02 14:41:44,762 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:41:06+00:00
2023-08-02 14:41:44,769 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-09-15 19:41:06+00:00
2023-08-02 14:41:44,813 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:44,818 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:44,912 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:44,918 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:44,922 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:44,927 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:44,932 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:44,937 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:44,941 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:44,945 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:44,952 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:44,957 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:44,961 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:44,966 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:44,970 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:44,975 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:44,988 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:44,992 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:44,995 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:45,000 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:45,006 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:45,010 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:45,014 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:45,018 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:45,023 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:45,028 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:45,032 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:45,037 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:45,041 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:45,046 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:45,051 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:45,055 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:45,061 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:45,067 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:45,072 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:45,077 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:45,081 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:45,086 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:45,089 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:45,093 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 19:27:46+00:00
2023-08-02 14:41:45,186 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-08-28 15:50:11+00:00
2023-08-02 14:41:45,189 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-08-28 15:50:11+00:00
2023-08-02 14:41:45,201 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-08-28 15:50:08+00:00
2023-08-02 14:41:45,204 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-08-28 15:50:08+00:00
2023-08-02 14:41:45,210 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-08-28 15:50:11+00:00
2023-08-02 14:41:45,214 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2023-08-28 15:50:11+00:00
2023-08-02 14:41:45,239 - ThreadPoolExecutor-3_0 ocsp_snowflake.py:585 - read_ocsp_response_cache_file() - DEBUG - Read OCSP response cache file: C:/Users/weldend/AppData/Local/Snowflake/Caches/ocsp_response_cache.json, count=188
2023-08-02 14:41:45,240 - ThreadPoolExecutor-3_0 ocsp_snowflake.py:1118 - validate() - DEBUG - validating certificate: someblob.blob.core.windows.net
2023-08-02 14:41:45,240 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:439 - extract_certificate_chain() - DEBUG - # of certificates: 2
2023-08-02 14:41:45,241 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:444 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('common_name', '*.blob.core.windows.net')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'Microsoft Corporation'), ('common_name', 'Microsoft RSA TLS CA 01')])
2023-08-02 14:41:45,243 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:444 - extract_certificate_chain() - DEBUG - subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Microsoft Corporation'), ('common_name', 'Microsoft RSA TLS CA 01')]), issuer: OrderedDict([('country_name', 'IE'), ('organization_name', 'Baltimore'), ('organizational_unit_name', 'CyberTrust'), ('common_name', 'Baltimore CyberTrust Root')])
2023-08-02 14:41:45,246 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:465 - create_pair_issuer_subject() - DEBUG - not found issuer_der: OrderedDict([('country_name', 'IE'), ('organization_name', 'Baltimore'), ('organizational_unit_name', 'CyberTrust'), ('common_name', 'Baltimore CyberTrust Root')])
2023-08-02 14:41:45,253 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2024-06-14 18:50:38+00:00
2023-08-02 14:41:45,254 - ThreadPoolExecutor-3_0 ocsp_snowflake.py:725 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('common_name', '*.blob.core.windows.net')])
2023-08-02 14:41:45,261 - ThreadPoolExecutor-3_0 ocsp_asn1crypto.py:238 - is_valid_time() - DEBUG - Verifying the attached certificate is signed by the issuer. Valid Not After: 2025-05-12 23:58:59+00:00
2023-08-02 14:41:45,263 - ThreadPoolExecutor-3_0 ocsp_snowflake.py:725 - find_cache() - DEBUG - hit cache for subject: OrderedDict([('country_name', 'US'), ('organization_name', 'Microsoft Corporation'), ('common_name', 'Microsoft RSA TLS CA 01')])
2023-08-02 14:41:45,269 - ThreadPoolExecutor-3_0 ocsp_snowflake.py:647 - write_ocsp_response_cache_file() - DEBUG - writing OCSP response cache file to C:/Users/weldend/AppData/Local/Snowflake/Caches/ocsp_response_cache.json
2023-08-02 14:41:45,270 - ThreadPoolExecutor-3_0 ocsp_snowflake.py:1794 - encode_ocsp_response_cache() - DEBUG - encoding OCSP response cache to JSON
2023-08-02 14:41:45,383 - ThreadPoolExecutor-3_0 ocsp_snowflake.py:1175 - _validate() - DEBUG - ok
2023-08-02 14:41:45,510 - ThreadPoolExecutor-3_0 connectionpool.py:456 - _make_request() - DEBUG - https://someblob.blob.core.windows.net:443 "HEAD /stageszz082d342b-399a-4f65-a33a-22e3c3da12c8/file0.txt?sv=2021-04-10&spr=https&se=2023-08-02T21%3A41%3A43Z&sr=c&sp=rwdl&sig=zf9vhd%2BvTZhlVFzvOjRdv%2FQx%2F7IyibQH8fgBdifvIEM%3D HTTP/1.1" 404 0
2023-08-02 14:41:45,511 - ThreadPoolExecutor-3_0 network.py:1152 - _use_requests_session() - DEBUG - Session status for SessionPool 'someblob.blob.core.windows.net', SessionPool 0/1 active sessions
2023-08-02 14:41:45,511 - ThreadPoolExecutor-3_0 storage_client.py:136 - get_digest() - DEBUG - getting digest file=C:\Users\weldend\AppData\Local\Temp\tmpyrm1wpy0\file0.txt
2023-08-02 14:41:45,525 - ThreadPoolExecutor-3_0 file_util.py:161 - get_digest_and_size_for_file() - DEBUG - getting digest and size: ib/yqdl6gOsuWp6uMFjeQTDxA7zLuKcvehaUemMvmRc=, 274745, file=C:\Users\weldend\AppData\Local\Temp\tmpyrm1wpy0\file0.txt
2023-08-02 14:41:45,525 - ThreadPoolExecutor-3_0 storage_client.py:219 - prepare_upload() - DEBUG - Preparing to upload C:\Users\weldend\AppData\Local\Temp\tmpyrm1wpy0\file0.txt
2023-08-02 14:41:45,526 - ThreadPoolExecutor-3_0 storage_client.py:152 - encrypt() - DEBUG - encrypting file=C:\Users\weldend\AppData\Local\Temp\tmpyrm1wpy0\file0.txt
2023-08-02 14:41:45,528 - ThreadPoolExecutor-3_0 encryption_util.py:157 - encrypt_file() - DEBUG - unencrypted file: C:\Users\weldend\AppData\Local\Temp\tmpyrm1wpy0\file0.txt, temp file: C:\Users\weldend\AppData\Local\Temp\tmpqq63qatn\file0.txt#ag3w34z4, tmp_dir: C:\Users\weldend\AppData\Local\Temp\tmpqq63qatn
2023-08-02 14:41:45,529 - ThreadPoolExecutor-3_0 encryption_util.py:76 - encrypt_stream() - DEBUG - key_size = 16
2023-08-02 14:41:45,537 - ThreadPoolExecutor-3_0 storage_client.py:225 - prepare_upload() - DEBUG - finished preprocessing
2023-08-02 14:41:45,537 - ThreadPoolExecutor-3_0 storage_client.py:230 - prepare_upload() - DEBUG - number of chunks 1
2023-08-02 14:41:45,537 - ThreadPoolExecutor-3_0 file_transfer_agent.py:450 - preprocess_done_cb() - DEBUG - Finished preparing file file0.txt
2023-08-02 14:41:45,541 - ThreadPoolExecutor-2_0 storage_client.py:404 - upload_chunk() - DEBUG - Uploading chunk 0 of file C:\Users\weldend\AppData\Local\Temp\tmpqq63qatn\file0.txt#ag3w34z4
2023-08-02 14:41:45,541 - ThreadPoolExecutor-2_0 network.py:1147 - _use_requests_session() - DEBUG - Session status for SessionPool 'someblob.blob.core.windows.net', SessionPool 1/1 active sessions
2023-08-02 14:41:45,541 - ThreadPoolExecutor-2_0 storage_client.py:277 - _send_request_with_retry() - DEBUG - storage client request with session <snowflake.connector.vendored.requests.sessions.Session object at 0x00000209342FD5A0>
2023-08-02 14:41:46,803 - ThreadPoolExecutor-2_0 connectionpool.py:456 - _make_request() - DEBUG - https://someblob.blob.core.windows.net:443 "PUT /stageszz082d342b-399a-4f65-a33a-22e3c3da12c8/file0.txt?sv=2021-04-10&spr=https&se=2023-08-02T21%3A41%3A43Z&sr=c&sp=rwdl&sig=zf9vhd%2BvTZhlVFzvOjRdv%2FQx%2F7IyibQH8fgBdifvIEM%3D HTTP/1.1" 201 0
2023-08-02 14:41:46,804 - ThreadPoolExecutor-2_0 network.py:1152 - _use_requests_session() - DEBUG - Session status for SessionPool 'someblob.blob.core.windows.net', SessionPool 0/1 active sessions
2023-08-02 14:41:46,804 - ThreadPoolExecutor-2_0 storage_client.py:406 - upload_chunk() - DEBUG - Successfully uploaded chunk 0 of file C:\Users\weldend\AppData\Local\Temp\tmpqq63qatn\file0.txt#ag3w34z4
2023-08-02 14:41:46,805 - ThreadPoolExecutor-2_0 file_transfer_agent.py:496 - transfer_done_cb() - DEBUG - Chunk 0/1 of file file0.txt reached callback
2023-08-02 14:41:46,805 - ThreadPoolExecutor-2_0 file_transfer_agent.py:512 - transfer_done_cb() - DEBUG - Chunk progress: file0.txt: completed: 1 failed: 0 total: 1
2023-08-02 14:41:46,805 - ThreadPoolExecutor-2_0 storage_client.py:246 - finish_upload() - DEBUG - C:\Users\weldend\AppData\Local\Temp\tmpyrm1wpy0\file0.txt upload is completed.
2023-08-02 14:41:46,806 - ThreadPoolExecutor-2_0 storage_client.py:440 - delete_client_data() - DEBUG - cleaning up tmp dir: C:\Users\weldend\AppData\Local\Temp\tmpqq63qatn
2023-08-02 14:41:46,811 - MainThread file_transfer_agent.py:705 - result() - DEBUG - raise_put_get_error: True, UPLOADED, <class 'str'>, ResultStatus.ERROR, <enum 'ResultStatus'>
2023-08-02 14:41:46,811 - MainThread cursor.py:841 - _init_result_and_meta() - DEBUG - Query result format: json
2023-08-02 14:41:46,812 - MainThread result_batch.py:440 - _parse() - DEBUG - parsing for result batch id: 1
2023-08-02 14:41:46,817 - MainThread pandas_tools.py:212 - write_pandas() - DEBUG - truncating table with 'TRUNCATE TABLE IF EXISTS "DCSKINPRODUCT" /* Python:snowflake.connector.pandas_tools.write_pandas() */ '
2023-08-02 14:41:46,817 - MainThread cursor.py:630 - execute() - DEBUG - executing SQL/command
2023-08-02 14:41:46,817 - MainThread cursor.py:669 - execute() - DEBUG - binding: [TRUNCATE TABLE IF EXISTS "DCSKINPRODUCT" /* Python:snowflake.connector.pandas_to...] with input=[None], processed=[{}]
2023-08-02 14:41:46,818 - MainThread cursor.py:715 - execute() - INFO - query: [TRUNCATE TABLE IF EXISTS "DCSKINPRODUCT" /* Python:snowflake.connector.pandas_to...]
2023-08-02 14:41:46,818 - MainThread connection.py:1302 - _next_sequence_counter() - DEBUG - sequence counter: 3
2023-08-02 14:41:46,818 - MainThread cursor.py:460 - _execute_helper() - DEBUG - Request id: a4157c4d-bbdc-4402-a4ef-0f7fbd4858f8
2023-08-02 14:41:46,818 - MainThread cursor.py:462 - _execute_helper() - DEBUG - running query [TRUNCATE TABLE IF EXISTS "DCSKINPRODUCT" /* Python:snowflake.connector.pandas_to...]
2023-08-02 14:41:46,819 - MainThread cursor.py:471 - _execute_helper() - DEBUG - is_file_transfer: False
2023-08-02 14:41:46,819 - MainThread connection.py:972 - cmd_query() - DEBUG - _cmd_query
2023-08-02 14:41:46,819 - MainThread connection.py:995 - cmd_query() - DEBUG - sql=[TRUNCATE TABLE IF EXISTS "DCSKINPRODUCT" /* Python:snowflake.connector.pandas_to...], sequence_id=[3], is_file_transfer=[False]
2023-08-02 14:41:46,820 - MainThread network.py:1147 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-08-02 14:41:46,820 - MainThread network.py:827 - _request_exec_wrapper() - DEBUG - remaining request timeout: None, retry cnt: 1
2023-08-02 14:41:46,821 - MainThread network.py:808 - add_request_guid() - DEBUG - Request guid: 34e0c5c7-2607-49ed-a3be-21601474d8cf
2023-08-02 14:41:46,821 - MainThread network.py:1006 - _request_exec() - DEBUG - socket timeout: 60
2023-08-02 14:41:47,131 - MainThread connectionpool.py:456 - _make_request() - DEBUG - https://myaccount.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=a4157c4d-bbdc-4402-a4ef-0f7fbd4858f8&request_guid=34e0c5c7-2607-49ed-a3be-21601474d8cf HTTP/1.1" 200 None
2023-08-02 14:41:47,133 - MainThread network.py:1032 - _request_exec() - DEBUG - SUCCESS
2023-08-02 14:41:47,134 - MainThread network.py:1152 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-08-02 14:41:47,134 - MainThread network.py:715 - _post_request() - DEBUG - ret[code] = None, after post request
2023-08-02 14:41:47,134 - MainThread network.py:739 - _post_request() - DEBUG - Query id: 01ae0b7d-0c04-d5f5-0000-09cd2834abee
2023-08-02 14:41:47,135 - MainThread cursor.py:737 - execute() - DEBUG - sfqid: 01ae0b7d-0c04-d5f5-0000-09cd2834abee
2023-08-02 14:41:47,135 - MainThread cursor.py:739 - execute() - INFO - query execution done
2023-08-02 14:41:47,136 - MainThread cursor.py:741 - execute() - DEBUG - SUCCESS
2023-08-02 14:41:47,136 - MainThread cursor.py:744 - execute() - DEBUG - PUT OR GET: None
2023-08-02 14:41:47,136 - MainThread cursor.py:841 - _init_result_and_meta() - DEBUG - Query result format: json
2023-08-02 14:41:47,136 - MainThread result_batch.py:440 - _parse() - DEBUG - parsing for result batch id: 1
2023-08-02 14:41:47,137 - MainThread pandas_tools.py:280 - write_pandas() - DEBUG - copying into with 'COPY INTO "DCSKINPRODUCT" /* Python:snowflake.connector.pandas_tools.write_pandas() */ ("PRODUCTKEY","DISTRIBUTIONCENTER","DATE","DISCOUNT","REVENUE") FROM (SELECT $1:"PRODUCTKEY",$1:"DISTRIBUTIONCENTER",$1:"DATE",$1:"DISCOUNT",$1:"REVENUE" FROM @"cbpzl") FILE_FORMAT=(TYPE=PARQUET COMPRESSION=auto) PURGE=TRUE ON_ERROR=abort_statement'
2023-08-02 14:41:47,137 - MainThread cursor.py:630 - execute() - DEBUG - executing SQL/command
2023-08-02 14:41:47,137 - MainThread cursor.py:669 - execute() - DEBUG - binding: [COPY INTO "DCSKINPRODUCT" /* Python:snowflake.connector.pandas_tools.write_panda...] with input=[None], processed=[{}]
2023-08-02 14:41:47,138 - MainThread cursor.py:715 - execute() - INFO - query: [COPY INTO "DCSKINPRODUCT" /* Python:snowflake.connector.pandas_tools.write_panda...]
2023-08-02 14:41:47,138 - MainThread connection.py:1302 - _next_sequence_counter() - DEBUG - sequence counter: 4
2023-08-02 14:41:47,138 - MainThread cursor.py:460 - _execute_helper() - DEBUG - Request id: ba0ca7c1-544b-4dbc-bfd6-cd029034ff8c
2023-08-02 14:41:47,138 - MainThread cursor.py:462 - _execute_helper() - DEBUG - running query [COPY INTO "DCSKINPRODUCT" /* Python:snowflake.connector.pandas_tools.write_panda...]
2023-08-02 14:41:47,139 - MainThread cursor.py:471 - _execute_helper() - DEBUG - is_file_transfer: False
2023-08-02 14:41:47,139 - MainThread connection.py:972 - cmd_query() - DEBUG - _cmd_query
2023-08-02 14:41:47,139 - MainThread connection.py:995 - cmd_query() - DEBUG - sql=[COPY INTO "DCSKINPRODUCT" /* Python:snowflake.connector.pandas_tools.write_panda...], sequence_id=[4], is_file_transfer=[False]
2023-08-02 14:41:47,140 - MainThread network.py:1147 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-08-02 14:41:47,140 - MainThread network.py:827 - _request_exec_wrapper() - DEBUG - remaining request timeout: None, retry cnt: 1
2023-08-02 14:41:47,141 - MainThread network.py:808 - add_request_guid() - DEBUG - Request guid: 9bcfa5da-39dd-4de6-b5a0-c38547d0f44f
2023-08-02 14:41:47,142 - MainThread network.py:1006 - _request_exec() - DEBUG - socket timeout: 60
2023-08-02 14:41:49,064 - MainThread connectionpool.py:456 - _make_request() - DEBUG - https://myaccount.snowflakecomputing.com:443 "POST /queries/v1/query-request?requestId=ba0ca7c1-544b-4dbc-bfd6-cd029034ff8c&request_guid=9bcfa5da-39dd-4de6-b5a0-c38547d0f44f HTTP/1.1" 200 None
2023-08-02 14:41:49,066 - MainThread network.py:1032 - _request_exec() - DEBUG - SUCCESS
2023-08-02 14:41:49,066 - MainThread network.py:1152 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-08-02 14:41:49,067 - MainThread network.py:715 - _post_request() - DEBUG - ret[code] = None, after post request
2023-08-02 14:41:49,067 - MainThread network.py:739 - _post_request() - DEBUG - Query id: 01ae0b7d-0c04-d5f7-0000-09cd28349ece
2023-08-02 14:41:49,068 - MainThread cursor.py:737 - execute() - DEBUG - sfqid: 01ae0b7d-0c04-d5f7-0000-09cd28349ece
2023-08-02 14:41:49,068 - MainThread cursor.py:739 - execute() - INFO - query execution done
2023-08-02 14:41:49,068 - MainThread cursor.py:741 - execute() - DEBUG - SUCCESS
2023-08-02 14:41:49,068 - MainThread cursor.py:744 - execute() - DEBUG - PUT OR GET: None
2023-08-02 14:41:49,069 - MainThread cursor.py:841 - _init_result_and_meta() - DEBUG - Query result format: json
2023-08-02 14:41:49,069 - MainThread result_batch.py:440 - _parse() - DEBUG - parsing for result batch id: 1
2023-08-02 14:41:49,070 - MainThread result_set.py:58 - result_set_iterator() - DEBUG - beginning to schedule result batch downloads
2023-08-02 14:41:49,070 - MainThread connection.py:557 - close() - INFO - closed
2023-08-02 14:41:49,070 - MainThread telemetry.py:151 - close() - DEBUG - Closing telemetry client.
2023-08-02 14:41:49,072 - MainThread telemetry.py:116 - send_batch() - DEBUG - Sending 1 logs to telemetry. Data is {'logs': [{'message': {'type': 'client_time_consume_first_result', 'source': 'PythonConnector', 'query_id': '01ae0b7d-0c04-d5f7-0000-09cd28349eb6', 'value': 72}, 'timestamp': '1691005303050'}, {'message': {'type': 'client_time_consume_last_result', 'source': 'PythonConnector', 'query_id': '01ae0b7d-0c04-d5f7-0000-09cd28349eb6', 'value': 2}, 'timestamp': '1691005303052'}, {'message': {'type': 'client_time_consume_first_result', 'source': 'PythonConnector', 'query_id': '01ae0b7d-0c04-d5f5-0000-09cd2834abee', 'value': 51}, 'timestamp': '1691005307135'}, {'message': {'type': 'client_time_consume_first_result', 'source': 'PythonConnector', 'query_id': '01ae0b7d-0c04-d5f7-0000-09cd28349ece', 'value': 59}, 'timestamp': '1691005309068'}, {'message': {'type': 'client_time_consume_last_result', 'source': 'PythonConnector', 'query_id': '01ae0b7d-0c04-d5f7-0000-09cd28349ece', 'value': 2}, 'timestamp': '1691005309070'}]}.
2023-08-02 14:41:49,073 - MainThread network.py:1147 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-08-02 14:41:49,074 - MainThread network.py:827 - _request_exec_wrapper() - DEBUG - remaining request timeout: 5, retry cnt: 1
2023-08-02 14:41:49,074 - MainThread network.py:808 - add_request_guid() - DEBUG - Request guid: e2e39818-44bd-479e-b5df-86c82081437c
2023-08-02 14:41:49,074 - MainThread network.py:1006 - _request_exec() - DEBUG - socket timeout: 60
2023-08-02 14:41:49,198 - MainThread connectionpool.py:456 - _make_request() - DEBUG - https://myaccount.snowflakecomputing.com:443 "POST /telemetry/send?request_guid=e2e39818-44bd-479e-b5df-86c82081437c HTTP/1.1" 200 86
2023-08-02 14:41:49,200 - MainThread network.py:1032 - _request_exec() - DEBUG - SUCCESS
2023-08-02 14:41:49,200 - MainThread network.py:1152 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-08-02 14:41:49,200 - MainThread network.py:715 - _post_request() - DEBUG - ret[code] = None, after post request
2023-08-02 14:41:49,201 - MainThread telemetry.py:140 - send_batch() - DEBUG - Successfully uploading metrics to telemetry.
2023-08-02 14:41:49,201 - MainThread connection.py:560 - close() - INFO - No async queries seem to be running, deleting session
2023-08-02 14:41:49,201 - MainThread network.py:1147 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 1/1 active sessions
2023-08-02 14:41:49,202 - MainThread network.py:827 - _request_exec_wrapper() - DEBUG - remaining request timeout: 5, retry cnt: 1
2023-08-02 14:41:49,202 - MainThread network.py:808 - add_request_guid() - DEBUG - Request guid: 15138651-74a5-4689-8873-78d5a38cbf4c
2023-08-02 14:41:49,202 - MainThread network.py:1006 - _request_exec() - DEBUG - socket timeout: 60
2023-08-02 14:41:49,569 - MainThread connectionpool.py:456 - _make_request() - DEBUG - https://myaccount.snowflakecomputing.com:443 "POST /session?delete=true&request_guid=15138651-74a5-4689-8873-78d5a38cbf4c HTTP/1.1" 200 76
2023-08-02 14:41:49,570 - MainThread network.py:1032 - _request_exec() - DEBUG - SUCCESS
2023-08-02 14:41:49,571 - MainThread network.py:1152 - _use_requests_session() - DEBUG - Session status for SessionPool 'myaccount.snowflakecomputing.com', SessionPool 0/1 active sessions
2023-08-02 14:41:49,571 - MainThread network.py:715 - _post_request() - DEBUG - ret[code] = None, after post request
2023-08-02 14:41:49,579 - MainThread connection.py:571 - close() - DEBUG - Session is closed
Initiating login request with your identity provider. A browser window should have opened for you to complete the login. If you can't see it, check existing browser windows, or your OS settings. Press CTRL+C to abort and try again...
152130 written to table DCSKINPRODUCT
2023-08-02 14:41:49,679 - MainThread connection.py:548 - close() - DEBUG - Rest object has been destroyed, cannot close session
dwelden commented 1 year ago

Forgot to include the sample data from the dataframe showing the date values expected:

>>> df
                                           ProductKey DistributionCenter       DATE  Discount  Revenue
0       Javier's Deep Cleansing Hair Shampoo 2.718 dl               Cary 2013-02-10       0.0      0.0
1       Javier's Deep Cleansing Hair Shampoo 2.718 dl               Cary 2013-02-17       0.0      0.0
2       Javier's Deep Cleansing Hair Shampoo 2.718 dl               Cary 2013-02-24       0.0      0.0
3       Javier's Deep Cleansing Hair Shampoo 2.718 dl               Cary 2013-03-03       0.0      0.0
4       Javier's Deep Cleansing Hair Shampoo 2.718 dl               Cary 2013-03-10       0.0      0.0
...                                               ...                ...        ...       ...      ...
152125            Sujatha's Sensitve skin Lotion 8 dl            Toronto 2015-02-15       0.0  27027.0
152126            Sujatha's Sensitve skin Lotion 8 dl            Toronto 2015-02-22       0.0  28124.0
152127            Sujatha's Sensitve skin Lotion 8 dl            Toronto 2015-03-01       0.0  25574.0
152128            Sujatha's Sensitve skin Lotion 8 dl            Toronto 2015-03-08       0.0  25812.0
152129            Sujatha's Sensitve skin Lotion 8 dl            Toronto 2015-03-15       0.0  25532.0

[152130 rows x 5 columns]
sfc-gh-sfan commented 1 year ago

Potentially a duplicate of https://github.com/snowflakedb/snowflake-connector-python/issues/600?

dwelden commented 1 year ago

It may be related to 600. I did not get the error message described there. It simply wrote invalid values to Snowflake.

sfc-gh-sfan commented 1 year ago

Are all data written as Invalid Date? Can I get a small dataset to reproduce?

dwelden commented 1 year ago

The data set used is DCSKINPRODUCT.sas7bdat from SAS-Visual-Forecasting---sample-data-sets

DCSKINPRODUCT.zip

-- Query Snowflake to examine the invalid dates
select to_varchar("DATE") from dcskinproduct limit 20;

43113053-09-24 17:00:00.000
43132219-01-30 16:00:00.000
43151384-06-05 17:00:00.000
43170549-10-11 17:00:00.000
43189715-02-16 16:00:00.000
43208880-06-22 17:00:00.000
43228045-10-28 17:00:00.000
43247211-03-05 16:00:00.000
43266376-07-10 17:00:00.000
43285541-11-15 16:00:00.000
43304707-03-23 17:00:00.000
43323872-07-27 17:00:00.000
43343037-12-02 16:00:00.000
43362203-04-09 17:00:00.000
43381368-08-13 17:00:00.000
43400533-12-19 16:00:00.000
43419699-04-25 17:00:00.000
43438864-08-30 17:00:00.000
43458030-01-05 16:00:00.000
43477195-05-13 17:00:00.000

The data type datetime64[ns] is stored internally as an int64 nanoseconds timedelta as of 1970-01-01. Given the Year values Snowflake has here, my suspicion is that the Snowflake connector is evaluating as if the timedelta were in milliseconds.

>>> df['DATE'][0], df['DATE'].astype('int64')[0]
(Timestamp('2013-02-10 00:00:00'), 1360454400000000000)
select to_varchar(timestampadd('milliseconds', 1360454400000000000, '1970-01-01'::timestamp));
43113053-09-25 00:00:00.000

select to_varchar(timestampadd('nanoseconds', 1360454400000000000, '1970-01-01'::timestamp));
2013-02-10 00:00:00.000
dwelden commented 1 year ago

For now, I have the following workaround in place, but this should not be required.

    # Map datetime columns to string (needed because of bug in write_pandas)
    for column in df.columns.to_list():
        if is_datetime64_dtype(df[column]):
            df[column] = df[column].dt.strftime('%Y-%m-%d %H:%M:%S.%f')
sfc-gh-sfan commented 1 year ago

I see. IIRC there is an ongoing rolling out of a feature that could impact how timestamp in parquet is interpreted, and write_pandas uses parquet file as a intermediate step. Let me check with @sfc-gh-yuliu .

sfc-gh-sfan commented 1 year ago

I confirmed with @sfc-gh-yuliu and it is related to the new logical type (nano second). There is an ongoing effort to rolling out this change and connector would need to support it with a new param (use_logical_type). Please stay tuned.

sfc-gh-stan commented 1 year ago

Duplicate of https://github.com/snowflakedb/snowpark-python/issues/991

sfc-gh-achandrasekaran commented 1 year ago

This has been fixed - please see linked ticket