apache / arrow-adbc

Database connectivity API standard and libraries for Apache Arrow
https://arrow.apache.org/adbc/
Apache License 2.0
357 stars 86 forks source link

JWT token is invalid with Snowflake Go Driver - Python #1422

Open davlee1972 opened 8 months ago

davlee1972 commented 8 months ago

I'm trying to connect via python to Snowflake using a P8 encrypted key file, but haven't had any success with either an encrypted or unencrypted file. I looked through the code and I didn't see many options to increase Go debugging/tracing to help me figure out what isn't working..

-----BEGIN ENCRYPTED PRIVATE KEY----- or -----BEGIN RSA PRIVATE KEY-----

With snowsql I've was able to use the rsa_key.p8 encrypted key file to successfully connect. I verified both the fingerprint and decrypted jwt token were also valid within Snowflake user settings and formatting:

snowsql --private-key-path rsa_key.p8 -a my_privatelink.snowflakecomputing.com -u MY_LOGIN -s MY_SCHEMA -d MY_DB -r MY_ROLE -w MY_WAREHOUSE -o log_level=DEBUG -o log_file=c:\temp\snow.log

Private Key Passphrase:
* SnowSQL * v1.2.30
Type SQL statements or !help
MY_LOGIN#MY_WAREHOUSE@MY_DB.MY_SCHEMA>

log file:
2024-01-02 19:25:21,160 (19000/MainThread) snowflake.connector.connection INFO     connection:1163 - This connection is in OCSP Fail Open Mode. TLS Certificates would be checked for validity and revocation status. Any other Certificate Revocation related exceptions or OCSP Responder failures would be disregarded in favor of connectivity.
2024-01-02 19:25:21,161 (19000/MainThread) snowflake.connector.converter DEBUG      converter:146  - use_numpy: False
2024-01-02 19:25:21,161 (19000/MainThread) snowflake.connector.connection DEBUG     connection:855  - REST API object was created: my_privatelink.snowflakecomputing.com:443
2024-01-02 19:25:21,162 (19000/MainThread) snowflake.connector.connection DEBUG     connection:835  - OCSP Cache Server is updated: http://ocsp.my_privatelink.snowflakecomputing.com/ocsp_response_cache.json
2024-01-02 19:25:21,209 (19000/MainThread) snowflake.connector.auth.keypair DEBUG        keypair:174  - Public key fingerprint is SHA256:qvmb???????etc.etc.etc.?????????????CYlU=
2024-01-02 19:25:21,213 (19000/MainThread) snowflake.connector.auth._auth DEBUG          _auth:174  - authenticate
2024-01-02 19:25:21,214 (19000/MainThread) snowflake.connector.auth._auth DEBUG          _auth:208  - assertion content: <bound method AuthByKeyPair.assertion_content of <snowflake.connector.auth.keypair.AuthByKeyPair object at 0x000002D87F0A15B0>>
2024-01-02 19:25:21,215 (19000/MainThread) snowflake.connector.auth._auth DEBUG          _auth:211  - account=my_account, user=MY_LOGIN, database=MY_DB, schema=MY_SCHEMA, warehouse=MY_WAREHOUSE, role=MY_ROLE, request_id=bbc25fe1-35a7-4c52-a889-7adc2d82108e
2024-01-02 19:25:21,215 (19000/MainThread) snowflake.connector.auth._auth DEBUG          _auth:244  - body['data']: {'CLIENT_APP_ID': 'SnowSQL', 'CLIENT_APP_VERSION': '1.2.30', 'SVN_REVISION': None, 'ACCOUNT_NAME': 'my_account', 'LOGIN_NAME': 'MY_LOGIN', 'CLIENT_ENVIRONMENT': {'APPLICATION': 'SnowSQL', 'OS': 'Windows', 'OS_VERSION': 'Windows-10-10.0.19045-SP0', 'PYTHON_VERSION': '3.8.18', 'PYTHON_RUNTIME': 'CPython', 'PYTHON_COMPILER': 'MSC v.1916 64 bit (AMD64)', 'OCSP_MODE': 'FAIL_OPEN', 'TRACING': 10, 'LOGIN_TIMEOUT': 120, 'NETWORK_TIMEOUT': None, 'SOCKET_TIMEOUT': None}, 'AUTHENTICATOR': 'SNOWFLAKE_JWT', 'TOKEN': '****.eyJpc?????????etc.etc.etc.???????????????tWTsw', 'SESSION_PARAMETERS': {'ABORT_DETACHED_QUERY': False, 'AUTOCOMMIT': True, 'CLIENT_SESSION_KEEP_ALIVE': False, 'CLIENT_PREFETCH_THREADS': 4}}
2024-01-02 19:25:21,218 (19000/MainThread) snowflake.connector.vendored.urllib3.util.retry DEBUG          retry:351  - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
2024-01-02 19:25:21,219 (19000/MainThread) snowflake.connector.vendored.urllib3.util.retry DEBUG          retry:351  - Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
2024-01-02 19:25:21,219 (19000/MainThread) snowflake.connector.network DEBUG        network:1217 - Session status for SessionPool 'my_privatelink.snowflakecomputing.com', SessionPool 1/1 active sessions
2024-01-02 19:25:21,220 (19000/MainThread) snowflake.connector.network DEBUG        network:868  - remaining request timeout: N/A ms, retry cnt: 1
2024-01-02 19:25:21,220 (19000/MainThread) snowflake.connector.network DEBUG        network:850  - Request guid: 6d34a410-3d28-4971-b325-3e332fa7aecb
2024-01-02 19:25:21,220 (19000/MainThread) snowflake.connector.network DEBUG        network:1058 - socket timeout: 10
2024-01-02 19:25:21,225 (19000/MainThread) snowflake.connector.vendored.urllib3.connectionpool DEBUG connectionpool:1003 - Starting new HTTPS connection (1): my_privatelink.snowflakecomputing.com:443
2024-01-02 19:25:22,254 (19000/MainThread) snowflake.connector.ssl_wrap_socket DEBUG ssl_wrap_socket:79   - OCSP Mode: FAIL_OPEN, OCSP response cache file name: None
2024-01-02 19:25:22,276 (19000/MainThread) snowflake.connector.ocsp_snowflake DEBUG ocsp_snowflake:492  - cache directory: C:\Users\MY_LOGIN\AppData\Local\Snowflake\Caches
2024-01-02 19:25:22,278 (19000/MainThread) snowflake.connector.ocsp_snowflake DEBUG ocsp_snowflake:530  - ocsp_response_cache_uri: file://C:/Users/MY_LOGIN/AppData/Local/Snowflake/Caches/ocsp_response_cache.json
2024-01-02 19:25:22,278 (19000/MainThread) snowflake.connector.ocsp_snowflake DEBUG ocsp_snowflake:533  - OCSP_VALIDATION_CACHE size: 208
2024-01-02 19:25:22,278 (19000/MainThread) snowflake.connector.ocsp_snowflake DEBUG ocsp_snowflake:332  - OCSP response cache server is enabled: http://ocsp.my_privatelink.snowflakecomputing.com/ocsp_response_cache.json
2024-01-02 19:25:22,279 (19000/MainThread) snowflake.connector.ocsp_snowflake DEBUG ocsp_snowflake:345  - OCSP dynamic cache server RETRY URL: http://ocsp.my_privatelink.snowflakecomputing.com/retry/{0}/{1}
2024-01-02 19:25:22,279 (19000/MainThread) snowflake.connector.ocsp_snowflake DEBUG ocsp_snowflake:966  - validating certificate: my_privatelink.snowflakecomputing.com
2024-01-02 19:25:22,280 (19000/MainThread) snowflake.connector.ocsp_asn1crypto DEBUG ocsp_asn1crypto:385  - # of certificates: 3
2024-01-02 19:25:22,281 (19000/MainThread) snowflake.connector.ocsp_asn1crypto DEBUG ocsp_asn1crypto:390  - subject: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'California'), ('locality_name', 'San Mateo'), ('organization_name', 'Snowflake Inc.'), ('common_name', '*.east-us-2.my_privatelink.snowflakecomputing.com')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('common_name', 'DigiCert Global G2 TLS RSA SHA256 2020 CA1')])
2024-01-02 19:25:22,283 (19000/MainThread) snowflake.connector.ocsp_asn1crypto DEBUG ocsp_asn1crypto:390  - subject: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('common_name', 'DigiCert Global G2 TLS RSA SHA256 2020 CA1')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('organizational_unit_name', 'www.digicert.com'), ('common_name', 'DigiCert Global Root G2')])
2024-01-02 19:25:22,285 (19000/MainThread) snowflake.connector.ocsp_asn1crypto DEBUG ocsp_asn1crypto:390  - subject: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('organizational_unit_name', 'www.digicert.com'), ('common_name', 'DigiCert Global Root G2')]), issuer: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('organizational_unit_name', 'www.digicert.com'), ('common_name', 'DigiCert Global Root G2')])
2024-01-02 19:25:22,290 (19000/MainThread) snowflake.connector.ocsp_snowflake DEBUG ocsp_snowflake:730  - hit cache for subject: OrderedDict([('country_name', 'US'), ('state_or_province_name', 'California'), ('locality_name', 'San Mateo'), ('organization_name', 'Snowflake Inc.'), ('common_name', '*.east-us-2.my_privatelink.snowflakecomputing.com')])
2024-01-02 19:25:22,294 (19000/MainThread) snowflake.connector.ocsp_snowflake DEBUG ocsp_snowflake:730  - hit cache for subject: OrderedDict([('country_name', 'US'), ('organization_name', 'DigiCert Inc'), ('common_name', 'DigiCert Global G2 TLS RSA SHA256 2020 CA1')])
2024-01-02 19:25:22,297 (19000/MainThread) snowflake.connector.ocsp_snowflake DEBUG ocsp_snowflake:1023 - ok
2024-01-02 19:25:22,494 (19000/MainThread) snowflake.connector.vendored.urllib3.connectionpool DEBUG connectionpool:456  - https://my_privatelink.snowflakecomputing.com:443 "POST /session/v1/login-request?request_id=bbc25fe1-35a7-4c52-a889-7adc2d82108e&databaseName=MY_DB&schemaName=MY_SCHEMA&warehouse=MY_WAREHOUSE&roleName=MY_ROLE&request_guid=6d34a410-3d28-4971-b325-3e332fa7aecb HTTP/1.1" 200 None
2024-01-02 19:25:22,496 (19000/MainThread) snowflake.connector.network DEBUG        network:1085 - SUCCESS
2024-01-02 19:25:22,496 (19000/MainThread) snowflake.connector.network DEBUG        network:1222 - Session status for SessionPool 'my_privatelink.snowflakecomputing.com', SessionPool 0/1 active sessions
2024-01-02 19:25:22,497 (19000/MainThread) snowflake.connector.network DEBUG        network:740  - ret[code] = None, after post request
2024-01-02 19:25:22,498 (19000/MainThread) snowflake.connector.auth._auth DEBUG          _auth:371  - completed authentication
2024-01-02 19:25:22,498 (19000/MainThread) snowflake.connector.auth._auth DEBUG          _auth:418  - token = ******
2024-01-02 19:25:22,498 (19000/MainThread) snowflake.connector.auth._auth DEBUG          _auth:424  - master_token = ******
2024-01-02 19:25:22,499 (19000/MainThread) snowflake.connector.auth._auth DEBUG          _auth:430  - id_token = NULL
2024-01-02 19:25:22,499 (19000/MainThread) snowflake.connector.auth._auth DEBUG          _auth:436  - mfa_token = NULL
2024-01-02 19:25:22,501 (19000/MainThread) snowflake.cli.sqlexecute DEBUG     sqlexecute:325  - adding connection: 0
2024-01-02 19:25:22,572 (19000/MainThread) snowflake.cli.clistyle DEBUG       clistyle:111  - prompt format: MY_LOGIN#MY_WAREHOUSE@MY_DB.MY_SCHEMA>, tokens: [(Token.TOKEN_0, 'MY_LOGIN#MY_WAREHOUSE@MY_DB.MY_SCHEMA>')]
2024-01-02 19:25:22,614 (19000/Thread-1) snowflake.connector.connection DEBUG     connection:774  - cursor
2024-01-02 19:25:22,615 (19000/Thread-2) snowflake.connector.connection DEBUG     connection:774  - cursor
2024-01-02 19:25:22,615 (19000/Thread-1) snowflake.connector.cursor DEBUG         cursor:766  - executing SQL/command
2024-01-02 19:25:22,616 (19000/Thread-3) snowflake.connector.connection DEBUG     connection:774  - cursor
2024-01-02 19:25:22,617 (19000/Thread-2) snowflake.connector.cursor DEBUG         cursor:766  - executing SQL/command
2024-01-02 19:25:22,618 (19000/Thread-4) snowflake.connector.connection DEBUG     connection:774  - cursor
2024-01-02 19:25:22,620 (19000/MainThread) __main__ DEBUG      eventloop:739  - Read from terminal

Passing the same rsa_key.p8 file with the decryption password using the following python code gives me an invalid JWT token error with limited debugging info:

with open("rsa_key.p8") as f:
    pemlines = f.read()

import adbc_driver_snowflake.dbapi

db_kwargs={
    "adbc.snowflake.sql.account": "MY_PRIVATELINK",
    "adbc.snowflake.sql.auth_type": "auth_jwt",
    "adbc.snowflake.sql.client_option.jwt_private_key_pkcs8_value": pemlines,
    "adbc.snowflake.sql.client_option.jwt_private_key_pkcs8_password": "my_decryption_password",
    "adbc.snowflake.sql.client_option.jwt_expire_timeout": "24h",
    "adbc.snowflake.sql.db": "MY_DB",
    "adbc.snowflake.sql.client_option.tracing": "DEBUG",
    "adbc.snowflake.sql.schema": "MY_SCHEMA",
    "adbc.snowflake.sql.warehouse": "MY_WAREHOUSE",
    "username": "MY_LOGIN",
    "adbc.snowflake.sql.role": "MY_ROLE",
}

a_conn = adbc_driver_snowflake.dbapi.connect(db_kwargs=db_kwargs)

Returns with these errors:

time="2024-01-02T11:18:26-08:00" level=info msg=OpenWithConfig func="gosnowflake.(*defaultLogger).Info" file="log.go:150"
time="2024-01-02T11:18:26-08:00" level=info msg="Authenticating via SNOWFLAKE_JWT" func="gosnowflake.(*defaultLogger).Infof" file="log.go:106"
time="2024-01-02T11:18:26-08:00" level=info msg="PARAMS for Auth: &map[databaseName:[MY_DB] roleName:[MY_ROLE] schemaName:[MY_SCHEMA] warehouse:[MY_WAREHOUSE]], https, my_privatelink.snowflakecomputing.com, 443, 1m0s, SNOWFLAKE_JWT" func=gosnowflake.authenticate file="auth.go:348"
time="2024-01-02T11:18:26-08:00" level=info msg="full URL: https://my_privatelink.snowflakecomputing.com:443/session/v1/login-request?databaseName=MY_DB&requestId=9831866f-ae88-43be-6934-2fba36eda91c&request_guid=007fcd50-7a72-4152-6653-de829f3e6ca9&roleName=MY_ROLE&schemaName=MY_SCHEMA&warehouse=MY_WAREHOUSE" func="gosnowflake.(*defaultLogger).Infof" file="log.go:106"
time="2024-01-02T11:18:26-08:00" level=info msg="retryHTTP.totalTimeout: 1m0s" func="gosnowflake.(*retryHTTP).execute" file="retry.go:218"
time="2024-01-02T11:18:26-08:00" level=debug msg="retry count: 0" func="gosnowflake.(*defaultLogger).Debugf" file="log.go:102"
time="2024-01-02T11:18:26-08:00" level=info msg="downloading OCSP Cache from server http://ocsp.my_privatelink.snowflakecomputing.com/ocsp_response_cache.json" func="gosnowflake.(*defaultLogger).Infof" file="log.go:106"
time="2024-01-02T11:18:26-08:00" level=info msg="retryHTTP.totalTimeout: 5s" func="gosnowflake.(*retryHTTP).execute" file="retry.go:218"
time="2024-01-02T11:18:26-08:00" level=debug msg="retry count: 0" func="gosnowflake.(*defaultLogger).Debugf" file="log.go:102"
time="2024-01-02T11:18:26-08:00" level=debug msg="StatusCode from OCSP Cache Server: 200\n" func="gosnowflake.(*defaultLogger).Debugf" file="log.go:102"
time="2024-01-02T11:18:26-08:00" level=debug msg="reading contents" func="gosnowflake.(*defaultLogger).Debugf" file="log.go:102"
time="2024-01-02T11:18:27-08:00" level=info msg="Subject: CN=*.east-us-2.privatelink.snowflakecomputing.com,O=Snowflake Inc.,L=San Mateo,ST=California,C=US, Issuer: CN=DigiCert Global G2 TLS RSA SHA256 2020 CA1,O=DigiCert Inc,C=US\n" func="gosnowflake.(*defaultLogger).Infof" file="log.go:106"
time="2024-01-02T11:18:27-08:00" level=info msg="Subject: CN=DigiCert Global G2 TLS RSA SHA256 2020 CA1,O=DigiCert Inc,C=US, Issuer: CN=DigiCert Global Root G2,OU=www.digicert.com,O=DigiCert Inc,C=US\n" func="gosnowflake.(*defaultLogger).Infof" file="log.go:106"
time="2024-01-02T11:18:27-08:00" level=info msg="writing OCSP Response cache file. C:\\Users\\MY_LOGIN\\AppData\\Local\\Snowflake\\Caches\\ocsp_response_cache.json\n" func="gosnowflake.(*defaultLogger).Infof" file="log.go:106"
time="2024-01-02T11:18:27-08:00" level=error msg="Authentication FAILED" func="gosnowflake.(*defaultLogger).Errorln" file="log.go:242"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\MY_LOGIN\Anaconda3\lib\site-packages\adbc_driver_snowflake\dbapi.py", line 120, in connect
    conn = adbc_driver_manager.AdbcConnection(db, **(conn_kwargs or {}))
  File "adbc_driver_manager\\_lib.pyx", line 617, in adbc_driver_manager._lib.AdbcConnection.__init__
  File "adbc_driver_manager\\_lib.pyx", line 227, in adbc_driver_manager._lib.check_error
adbc_driver_manager.OperationalError: IO: 390144 (08004): JWT token is invalid. [0eb0d647-d2e3-40f9-b680-2f6181d9e699]
davlee1972 commented 8 months ago

I was also able to use the python snowflake connector successfully with the same rsa_key.p8 file..

https://docs.snowflake.com/en/developer-guide/python-connector/python-connector-connect

import snowflake.connector
import os
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.primitives.asymmetric import dsa
from cryptography.hazmat.primitives import serialization
with open("<path>/rsa_key.p8", "rb") as key:
    p_key= serialization.load_pem_private_key(
        key.read(),
        password=os.environ['PRIVATE_KEY_PASSPHRASE'].encode(),
        backend=default_backend()
    )

pkb = p_key.private_bytes(
    encoding=serialization.Encoding.DER,
    format=serialization.PrivateFormat.PKCS8,
    encryption_algorithm=serialization.NoEncryption())

ctx = snowflake.connector.connect(
    user='<user>',
    account='<account_identifier>',
    private_key=pkb,
    warehouse=WAREHOUSE,
    database=DATABASE,
    schema=SCHEMA
    )

cs = ctx.cursor()
davlee1972 commented 8 months ago

After upgrading to adbc 0.9.0, I'm seeing a different error.

Traceback (most recent call last): File "", line 1, in File "\Anaconda3\lib\site-packages\adbc_driver_snowflake\dbapi.py", line 120, in connect conn = adbc_driver_manager.AdbcConnection(db, **(conn_kwargs or {})) File "adbc_driver_manager\_lib.pyx", line 676, in adbc_driver_manager._lib.AdbcConnection.init File "adbc_driver_manager\_lib.pyx", line 232, in adbc_driver_manager._lib.check_error adbc_driver_manager.OperationalError: IO: 394300 (08004): !394300!

https://docs.snowflake.com/en/user-guide/key-pair-auth-troubleshooting#label-key-pair-troubleshooting-common-errors

394300 | JWT_TOKEN_INVALID_USER_IN_ISSUER | The user name specified in the issuer does not exist in the Snowflake account. For possible solutions, see Common Errors and Solutions.

This is even more odd. It looks like JWT is working now, but the ADBC "username" is not a valid snowflake "LOGIN_NAME".

Will try to trace this..

lidavidm commented 8 months ago

@zeroshade any ideas?

zeroshade commented 8 months ago

@davlee1972 does the username you're using for login match the same user that the JWT was issued for?

In theory, based on that error message, it looks like the "username" is being seen as valid just fine, but the "issuer" on the JWT is the user name that is not being recognized. That said, I'm not sure why it's working for the snowflake python connector and not working for this. I'll see if i can try to replicate the issue on my end but i need to bug my own snowflake account admins to give me permissions to create RSA key stuff.

davlee1972 commented 7 months ago

Here's some additional debugging using SNOWSQL to generate a JWT token and using it to connect.. I mentioned above, the debugging options for the JWT token including the issuer generated by the ADBC / GO driver code are limited and don't show the JWT token value being generated..

Using https://jwt.io/ I verified that the jwt token used by SNOWSQL is:

"iss": "<my_account>.<my_login>.SHA256:qvmbDr2 etc.. etc.. etc.. Fs+1CYlU=", which is account.login.fingerprint..

Testing steps:

  1. Use SNOWSQL to connect to Snowflake using encrypted rsa private key file.
  2. Generate a JWT token using SNOWSQL
  3. Try connecting using ADBC using the JWT token generated as a file..
snowsql --private-key-path rsa_key.p8 -a <my_account>.privatelink -u <my_login>
Private Key Passphrase:
* SnowSQL * v1.2.30
Type SQL statements or !help

snowsql --private-key-path rsa_key.p8 -a <my_account>.privatelink -u <my_login> --generate-jwt
Enter private key file password (Press enter if none) []:
eyJhbGciOiJSUzI1Ni
etc.. etc.. etc..
i86gxRk7w7QOTR8QvoxwQ
>>> import adbc_driver_snowflake.dbapi
>>>
>>> jwt_token = "eyJhbGciOiJSUzI1Ni etc.. etc.. etc.. i86gxRk7w7QOTR8QvoxwQ"
>>>
>>> db_kwargs={
...     "adbc.snowflake.sql.account": "<my_account>",
...     "adbc.snowflake.sql.auth_type": "auth_jwt",
...     "adbc.snowflake.sql.client_option.auth_token": jwt_token,
...     "adbc.snowflake.sql.client_option.jwt_expire_timeout": "24h",
...     "adbc.snowflake.sql.client_option.tracing": "DEBUG",
...     "username": "<my_login>",
... }
>>>
>>> a_conn = adbc_driver_snowflake.dbapi.connect(db_kwargs=db_kwargs)
time="2024-01-16T10:39:28-08:00" level=info msg=OpenWithConfig func="gosnowflake.(*defaultLogger).Info" file="log.go:216"
time="2024-01-16T10:39:28-08:00" level=debug msg="OCSP Cache Server for Privatelink: http://ocsp.<my_account>.privatelink.snowflakecomputing.com/ocsp_response_cache.json\n" func="gosnowflake.(*defaultLogger).Debugf" file="log.go:148"
time="2024-01-16T10:39:28-08:00" level=debug msg="OCSP Retry URL for Privatelink: http://ocsp.<my_account>.privatelink.snowflakecomputing.com/retry/%v/%v\n" func="gosnowflake.(*defaultLogger).Debugf" file="log.go:148"
time="2024-01-16T10:39:28-08:00" level=info msg="Authenticating via SNOWFLAKE_JWT" func="gosnowflake.(*defaultLogger).Infof" file="log.go:154"
time="2024-01-16T10:39:28-08:00" level=info msg="PARAMS for Auth: &map[], https, <my_account>.privatelink.snowflakecomputing.com, 443, 5m0s, SNOWFLAKE_JWT" func=gosnowflake.authenticate file="auth.go:353"
time="2024-01-16T10:39:28-08:00" level=info msg="full URL: https://<my_account>.privatelink.snowflakecomputing.com:443/session/v1/login-request?requestId=ea8252a2-64fe-4526-4b72-e2024078d4aa&request_guid=eca249bc-5f1a-4172-4250-19931c223f89" func="gosnowflake.(*defaultLogger).Infof" file="log.go:154"
time="2024-01-16T10:39:28-08:00" level=info msg="retryHTTP.totalTimeout: 5m0s" func="gosnowflake.(*retryHTTP).execute" file="retry.go:294"
time="2024-01-16T10:39:28-08:00" level=debug msg="retry count: 0" func="gosnowflake.(*defaultLogger).Debugf" file="log.go:148"
Snowflake driver panicked, stack traces:
goroutine 17 [running, locked to thread]:
main.poison(0x0?, {0x7ffd4b1c2dbc, 0x12}, {0x7ffd4afc7e80, 0x7ffd4ae5e610})
        D:/a/arrow-adbc/arrow-adbc/adbc/go/adbc/pkg/snowflake/driver.go:165 +0x87
main.SnowflakeConnectionInit.func1()
        D:/a/arrow-adbc/arrow-adbc/adbc/go/adbc/pkg/snowflake/driver.go:952 +0x51
panic({0x7ffd4afc7e80, 0x7ffd4ae5e610})
        C:/hostedtoolcache/windows/go/1.20.8/x64/src/runtime/panic.go:884 +0x213
crypto/rsa.(*PrivateKey).Public(...)
        C:/hostedtoolcache/windows/go/1.20.8/x64/src/crypto/rsa/rsa.go:123
github.com/snowflakedb/gosnowflake.prepareJWTToken(0xc0000c2e00)
        C:/Users/runneradmin/go/pkg/mod/github.com/snowflakedb/gosnowflake@v1.6.23-0.20231106142408-8445dcaf46d9/auth.go:462 +0x38
github.com/snowflakedb/gosnowflake.createRequestBody(0xc0006b1b60, 0xc000090200?, {{0x7ffd4b1b3bab, 0x2}, {0x7ffd4b1b8393, 0x7}, {0xc0000c6158, 0x8}, {0x7ffd4b1b95bf, 0x9}}, ...)
        C:/Users/runneradmin/go/pkg/mod/github.com/snowflakedb/gosnowflake@v1.6.23-0.20231106142408-8445dcaf46d9/auth.go:425 +0x429
github.com/snowflakedb/gosnowflake.authenticate.func1()
        C:/Users/runneradmin/go/pkg/mod/github.com/snowflakedb/gosnowflake@v1.6.23-0.20231106142408-8445dcaf46d9/auth.go:336 +0x6d
github.com/snowflakedb/gosnowflake.(*retryHTTP).execute(0xc0005067e8)
        C:/Users/runneradmin/go/pkg/mod/github.com/snowflakedb/gosnowflake@v1.6.23-0.20231106142408-8445dcaf46d9/retry.go:305 +0x209
github.com/snowflakedb/gosnowflake.postAuthRestful({0x7ffd4b40def8?, 0xc00092cb40?}, 0x19?, 0xc000911190?, 0x7ffd4af273a0?, 0xc0000a8000?, 0xc000491f58?, 0xc000491f30?)
        C:/Users/runneradmin/go/pkg/mod/github.com/snowflakedb/gosnowflake@v1.6.23-0.20231106142408-8445dcaf46d9/restful.go:197 +0x10f
github.com/snowflakedb/gosnowflake.postAuth({0x7ffd4b40def8, 0xc00092cb40}, 0xc0000c5040, 0x22?, 0xc00000a0f8, 0xc000506b00?, 0x7ffd4a94905a?, 0xc000491e30?)
        C:/Users/runneradmin/go/pkg/mod/github.com/snowflakedb/gosnowflake@v1.6.23-0.20231106142408-8445dcaf46d9/auth.go:229 +0x34d
github.com/snowflakedb/gosnowflake.authenticate({0x7ffd4b40def8, 0xc00092cb40}, 0xc0006b1b60, {0x0, 0x0, 0x0}, {0x0, 0x0, 0x0})
        C:/Users/runneradmin/go/pkg/mod/github.com/snowflakedb/gosnowflake@v1.6.23-0.20231106142408-8445dcaf46d9/auth.go:356 +0xf8e
github.com/snowflakedb/gosnowflake.authenticateWithConfig(0xc0006b1b60)
        C:/Users/runneradmin/go/pkg/mod/github.com/snowflakedb/gosnowflake@v1.6.23-0.20231106142408-8445dcaf46d9/auth.go:547 +0x398
github.com/snowflakedb/gosnowflake.SnowflakeDriver.OpenWithConfig({}, {_, _}, {{0xc0008c47c0, 0x20}, {0xc0008cf920, 0x4}, {0x0, 0x0}, {0x0, ...}, ...})
        C:/Users/runneradmin/go/pkg/mod/github.com/snowflakedb/gosnowflake@v1.6.23-0.20231106142408-8445dcaf46d9/driver.go:43 +0x15e
github.com/snowflakedb/gosnowflake.Connector.Connect({{0x7ffd4b400678, 0x7ffd4bf0bc78}, {{0xc0008c47c0, 0x20}, {0xc0008cf920, 0x4}, {0x0, 0x0}, {0x0, 0x0}, ...}}, ...)
        C:/Users/runneradmin/go/pkg/mod/github.com/snowflakedb/gosnowflake@v1.6.23-0.20231106142408-8445dcaf46d9/connector.go:34 +0x105
github.com/apache/arrow-adbc/go/adbc/driver/snowflake.(*databaseImpl).Open(0xc000923080, {0x7ffd4b40de88?, 0xc000038070?})
        D:/a/arrow-adbc/arrow-adbc/adbc/go/adbc/driver/snowflake/snowflake_database.go:454 +0x1f9
github.com/apache/arrow-adbc/go/adbc/driver/driverbase.(*database).Open(0x195b0446480?, {0x7ffd4b40de88?, 0xc000038070?})
        D:/a/arrow-adbc/arrow-adbc/adbc/go/adbc/driver/driverbase/database.go:134 +0x2b
main.SnowflakeConnectionInit(0x195b0493c78, 0x0?, 0xbd57eeee8)
        D:/a/arrow-adbc/arrow-adbc/adbc/go/adbc/pkg/snowflake/driver.go:968 +0x118
Traceback (most recent call last):
  File "C:\Users\<my_login>\Anaconda3\lib\site-packages\adbc_driver_snowflake\dbapi.py", line 120, in connect
    conn = adbc_driver_manager.AdbcConnection(db, **(conn_kwargs or {}))
  File "adbc_driver_manager\\_lib.pyx", line 676, in adbc_driver_manager._lib.AdbcConnection.__init__
  File "adbc_driver_manager\\_lib.pyx", line 232, in adbc_driver_manager._lib.check_error
adbc_driver_manager.InternalError: INTERNAL: [Snowflake] AdbcConnectionInit: Go panic in Snowflake driver (see stderr): "invalid memory address or nil pointer dereference"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\<my_login>\Anaconda3\lib\site-packages\adbc_driver_snowflake\dbapi.py", line 126, in connect
    db.close()
  File "adbc_driver_manager\\_lib.pyx", line 470, in adbc_driver_manager._lib.AdbcDatabase.close
  File "adbc_driver_manager\\_lib.pyx", line 476, in adbc_driver_manager._lib.AdbcDatabase.close
  File "adbc_driver_manager\\_lib.pyx", line 232, in adbc_driver_manager._lib.check_error
adbc_driver_manager.ProgrammingError: INVALID_STATE: [Snowflake] AdbcDatabaseRelease: Go panicked, driver is in unknown state
zeroshade commented 7 months ago

I've been away for a bit, but I just got my RSA key stuff set up so I can start trying to debug this myself. I should hopefully have an update tomorrow

davlee1972 commented 7 months ago

I also tried passing in adbc.snowflake.sql.account": " vs adbc.snowflake.sql.account": ".privatelink" The later example is above.. The prior example just cycles through RETRIES because the account url isn't valid.. time="2024-01-16T10:53:53-08:00" level=warning msg="failed http connection. err: Post \"https://<my_account>.snowflakecomputing.com:443/

davlee1972 commented 7 months ago

I think the problem might be this "privatelink" string in the driver code..

The correct iss should not include "privatelink" text.. "iss": "<my_account>.<my_login>.SHA256:qvmbDr2 etc.. etc.. etc.. Fs+1CYlU=",

However the account url has to have "privatelink" in it..

snowsql --private-key-path rsa_key.p8 -a <my_account>.privatelink -u MY_LOGIN -s MY_SCHEMA -d MY_DB -r MY_ROLE -w MY_WAREHOUSE -o log_level=DEBUG -o log_file=c:\temp\snow.log

For consistency with both SNOWSQL and the Snowflake Python Connector lib, ,the account should be in format and ".privatelink" might need to be stripped out when passing just to the GO driver?

davlee1972 commented 7 months ago

Ok the ".privatelink" is the problem.. it needs to be addressed when generating the host url string..

I got it to work using the following combination..

"adbc.snowflake.sql.account": "<my_account_without_privatelink>",
"adbc.snowflake.sql.uri.host": "<my_account_with_privatelink>".snowflakecomputing.com"

For consistency with other Snowflake clients the account should be: "adbc.snowflake.sql.account": "", << ADBC code should strip ".privatelink" before passing it to the GO driver.. "adbc.snowflake.sql.uri.host": "".snowflakecomputing.com"

zeroshade commented 7 months ago

As far as I can tell from the docs, the .privatelink suffix is only for use with AWS PrivateLink. Where in the docs does it say you should use the .privatelink when specifying the account? If this relies on the .privatelink suffix, then I likely won't be able to replicate the issue on my end as my configuration doesn't utilize AWS PrivateLink.

davlee1972 commented 7 months ago

The account param is used to generate the default uri so in my case I have to pass in .privatelink as part of the account.

But the jwt issuer is just account. Can we strip .privatelink from the account param when generating the jwt?

BTW using account with .privatelink works fine when using adbc with auth external instead of jwt.

zeroshade commented 7 months ago

if the JWT issuer is just account, it feels like we shouldn't have ADBC strip the .privatelink but rather it should be explicitly stated that the "account" provided has to match the issuer for the JWT. I don't want to add stripping of the .privatelink because what if the JWT issuer actually does have the .privatelink on it?

davlee1972 commented 7 months ago

This could be an issue with the go driver or handled in adbc.. Here's the equivalent snowflake python connector code..

https://github.com/snowflakedb/snowflake-connector-python/blob/main/src/snowflake/connector/util_text.py

host in my case needs to be passed in with account = my_account.privatelink host = f"{account}.snowflakecomputing.com"

the parsed_account turns my_account.privatelink to just my_account which essentially strips ".privatelink" url_parts = account.split(".") parsed_account = url_parts[0]

def construct_hostname(region: str | None, account: str) -> str:
    """Constructs hostname from region and account."""
    if region == "us-west-2":
        region = ""
    if region:
        if account.find(".") > 0:
            account = account[0 : account.find(".")]
        host = f"{account}.{region}.snowflakecomputing.com"
    else:
        host = f"{account}.snowflakecomputing.com"
    return host

def parse_account(account):
    url_parts = account.split(".")
    # if this condition is true, then we have some extra
    # stuff in the account field.
    if len(url_parts) > 1:
        if url_parts[1] == "global":
            # remove external ID from account
            parsed_account = url_parts[0][0 : url_parts[0].rfind("-")]
        else:
            # remove region subdomain
            parsed_account = url_parts[0]
    else:
        parsed_account = account

    return parsed_account
lidavidm commented 7 months ago

If something is to remove the privatelink part, it should be the user or gosnowflake; adbc is just forwarding the values in this case and it doesn't make sense to patch around gosnowflake's behavior

davlee1972 commented 7 months ago

Yeah and here's the valid JWT test which clearly strips stuff after "." unless it is ".GLOBAL" It looks "-" could also be problematic.

https://github.com/snowflakedb/snowflake-connector-python/blob/25f2547f11f17701f54ac6409cea9c764a0027b3/test/integ/test_key_pair_authentication.py#L12

@pytest.mark.parametrize(
    "input_account,expected_account",
    [
        ("s3testaccount.global", "S3TESTACCOUNT.GLOBAL"),
        ("acct-with-dashes", "ACCT-WITH-DASHES"),
        ("testaccount.extra", "TESTACCOUNT"),
        ("testaccount-user.global", "TESTACCOUNT"),
        ("normalaccount", "NORMALACCOUNT"),
    ],
)
zeroshade commented 7 months ago

@davlee1972 Can you file an issue on https://github.com/snowflakedb/gosnowflake referencing this. I agree with @lidavidm that this should either be handled by the user or by gosnowflake, not at the ADBC level.

davlee1972 commented 7 months ago

I'll open a ticket with Snowflake. Using a single account for both the URL and JWT won't work. This should be handled by the Snowflake GO driver just like how they handle it with the Snowflake PYTHON driver.