snowflakedb / snowflake-sqlalchemy

Snowflake SQLAlchemy
https://pypi.python.org/pypi/snowflake-sqlalchemy/
Apache License 2.0
228 stars 148 forks source link

SNOW-1511108: client_session_keep_alive_heartbeat_frequency is parsed as string instead of int #509

Open aditya81070 opened 3 weeks ago

aditya81070 commented 3 weeks ago

Please answer these questions before submitting your issue. Thanks!

  1. What version of Python are you using?
Python 3.11.9 (main, Apr  2 2024, 08:25:04) [Clang 15.0.0 (clang-1500.3.9.4)]
  1. What operating system and processor architecture are you using?
macOS-14.4.1-arm64-arm-64bit
  1. What are the component versions in the environment (pip freeze)?
asn1crypto==1.5.1
certifi==2024.6.2
cffi==1.16.0
charset-normalizer==3.3.2
cryptography==42.0.8
filelock==3.15.4
idna==3.7
numpy==2.0.0
packaging==24.1
panda==0.3.1
pandas==2.2.2
platformdirs==4.2.2
pycparser==2.22
PyJWT==2.8.0
pyOpenSSL==24.1.0
python-dateutil==2.9.0.post0
pytz==2024.1
requests==2.32.3
six==1.16.0
snowflake-connector-python==3.11.0
snowflake-sqlalchemy==1.5.3
sortedcontainers==2.4.0
SQLAlchemy==1.4.52
tomlkit==0.12.5
typing_extensions==4.12.2
tzdata==2024.1
  1. What did you do? I am trying to establish a connection to Snowflake with Keep Alive properties. I am trying to set the following two properties:
  2. client_session_keep_alive
  3. client_session_keep_alive_heartbeat_frequency

If I only set client_session_keep_alive, the connection is successful and I can execute the queries. If I set client_session_keep_alive_heartbeat_frequency, I get the following error:

'>' not supported between instances of 'str' and 'int'

Example code (not working)

from sqlalchemy import create_engine
engine = create_engine(
      'snowflake://{account_identifier}/{database}?client_session_keep_alive=True&client_session_keep_alive_heartbeat_frequency={client_session_keep_alive_heartbeat_frequency}&authenticator=OAUTH&role={role}&token={token}&warehouse={warehouse}'.format(
          account_identifier=account_identifier,
          database=database,
          role=role,
          token=access_token,
          warehouse=warehouse,
          client_session_keep_alive_heartbeat_frequency=900
      )
  )
connection = engine.connect()
results = connection.execute('SELECT SYSTEM$WAIT({}) AS delay'.format(duration)).fetchone()

I tried the same session parameters with snowflake python connector and it is working as expected.

import snowflake.connector

connection = snowflake.connector.connect(
  account=account_identifier,
  database=database,
  role=role,
  token=access_token,
  warehouse=warehouse,
  authenticator='OAUTH',
  session_parameters = {
    'CLIENT_SESSION_KEEP_ALIVE': True,
    'CLIENT_SESSION_KEEP_ALIVE_HEARTBEAT_FREQUENCY': 900
  }
)
  1. What did you expect to see?

The connection to Snowflake should be successful with the correct session configuration. But connection failed.

  1. Can you set logging to DEBUG and collect the logs?
2024-06-28 11:21:07,328 - MainThread ssl_wrap_socket.py:44 - inject_into_urllib3() - DEBUG - Injecting ssl_wrap_socket_with_ocsp
2024-06-28 11:21:07,328 - MainThread _auth.py:91 - <module>() - DEBUG - cache directory: /Users/aditya.agarwal/Library/Caches/Snowflake
2024-06-28 11:21:07,346 - MainThread cursor.py:87 - <module>() - DEBUG - Failed to import pyarrow or pandas. Cannot use pandas fetch API. Please install snowflake-connector-python with the `pandas` extra to use these features.
Testing long running queries with durations [14405] (in seconds).
Trying to connect to Snowflake.
2024-06-28 11:21:07,371 - MainThread connection.py:408 - __init__() - INFO - Snowflake Connector for Python Version: 3.11.0, Python Version: 3.11.9, Platform: macOS-14.4.1-arm64-arm-64bit
2024-06-28 11:21:07,371 - MainThread connection.py:714 - connect() - DEBUG - connect
2024-06-28 11:21:07,371 - MainThread connection.py:1099 - __config() - DEBUG - __config
2024-06-28 11:21:07,371 - MainThread connection.py:1258 - __config() - INFO - This connection is in OCSP Fail Open Mode. TLS Certificates would be checked for validity and revocation status. Any other Certificate Revocation related exceptions or OCSP Responder failures would be disregarded in favor of connectivity.
2024-06-28 11:21:07,371 - MainThread converter.py:159 - __init__() - DEBUG - use_numpy: False
2024-06-28 11:21:07,371 - MainThread connection.py:926 - __open_connection() - DEBUG - REST API object was created: company.snowflakecomputing.com:443
Error occurred while executing the query. Exiting the test.
'>' not supported between instances of 'str' and 'int'
sfc-gh-dszmolka commented 2 weeks ago

hi and thank you for raising this with us. the particular value seems to be passed as a str indeed , into the underlying PythonConnector and the underlying validation mechanism bails out.

I created a PR in the PythonConnector (https://github.com/snowflakedb/snowflake-connector-python/pull/1993) which hopefully fixes this issue; at least it did in the manual local tests.

sfc-gh-dszmolka commented 4 days ago

https://github.com/snowflakedb/snowflake-connector-python/pull/1993/ is merged, and will be part of the next PythonConnector release