sfc-gh-aling
commented
1 year ago hi @hsheth2 , thanks for reaching out. We had a PR out: https://github.com/snowflakedb/snowflake-connector-python/pull/1409 for fixing the issue, also thanks for the detailed explanation you've put in your patch code.
sfc-gh-achandrasekaran
Please answer these questions before submitting your issue. Thanks!
macOS-10.16-x86_64-i386-64bitpip freeze)?Env
``` absl-py==1.3.0 -e git+ssh://git@github.com/datahub-project/datahub.git@d2ff513bf4b1a1fdfdd7b7b87cfdf0b8ad7d2cca#egg=acryl_datahub&subdirectory=metadata-ingestion -e git+ssh://git@github.com/datahub-project/datahub.git@d2ff513bf4b1a1fdfdd7b7b87cfdf0b8ad7d2cca#egg=acryl_datahub_airflow_plugin&subdirectory=metadata-ingestion-modules/airflow-plugin acryl-datahub-classify==0.0.4 acryl-iceberg-legacy==0.0.4 acryl-PyHive==0.6.13 aenum==3.1.11 aiohttp==3.8.3 aiosignal==1.3.1 alembic==1.9.1 altair==4.2.0 anyio==3.6.2 apache-airflow==2.5.0 apache-airflow-providers-common-sql==1.3.2 apache-airflow-providers-ftp==3.3.0 apache-airflow-providers-http==4.1.0 apache-airflow-providers-imap==3.1.1 apache-airflow-providers-snowflake==4.0.2 apache-airflow-providers-sqlite==3.3.1 apispec==3.3.2 appdirs==1.4.4 appnope==0.1.3 argcomplete==2.0.0 argon2-cffi==21.3.0 argon2-cffi-bindings==21.2.0 asn1crypto==1.5.1 asttokens==2.2.1 async-timeout==4.0.2 attrs==22.2.0 Authlib==1.2.0 avro==1.10.2 -e git+https://github.com/acryldata/avro_gen@59fc21f5c17ce86a63cb1aeac12b403c093f6dad#egg=avro_gen3 azure-core==1.26.2 azure-identity==1.10.0 azure-storage-blob==12.14.1 azure-storage-file-datalake==12.9.1 Babel==2.11.0 backcall==0.2.0 backoff==2.2.1 bcrypt==4.0.1 beautifulsoup4==4.11.1 black==22.12.0 bleach==5.0.1 blinker==1.5 blis==0.7.9 boto3==1.26.46 boto3-stubs==1.26.46 botocore==1.29.46 botocore-stubs==1.29.46 bowler==0.9.0 bracex==2.3.post1 build==0.9.0 cached-property==1.5.2 cachelib==0.10.0 cachetools==5.2.1 catalogue==2.0.8 cattrs==22.2.0 certifi==2022.12.7 cffi==1.15.1 chardet==5.1.0 charset-normalizer==2.1.1 click==8.1.3 click-default-group==1.2.2 click-spinner==0.1.10 clickclick==20.10.2 clickhouse-driver==0.2.5 clickhouse-sqlalchemy==0.2.3 cloudpickle==2.2.0 colorama==0.4.6 colorlog==4.8.0 commonmark==0.9.1 confection==0.0.4 ConfigUpdater==3.1.1 confluent-kafka==1.9.2 connexion==2.14.1 coverage==7.0.4 cron-descriptor==1.2.32 croniter==1.3.8 cryptography==37.0.4 cx-Oracle==8.3.0 cymem==2.0.7 dask==2022.1.1 databricks-cli==0.17.3 databricks-dbapi==0.6.0 debugpy==1.6.5 decorator==5.1.1 deepdiff==6.2.3 defusedxml==0.7.1 deltalake==0.6.3 Deprecated==1.2.13 dill==0.3.6 distlib==0.3.6 distro==1.8.0 dnspython==2.2.1 docker==6.0.1 dockerpty==0.4.1 docopt==0.6.2 docutils==0.19 duckdb==0.6.1 ecdsa==0.18.0 elasticsearch==7.13.4 email-validator==1.3.0 entrypoints==0.4 et-xmlfile==1.1.0 exceptiongroup==1.1.0 executing==1.2.0 expandvars==0.9.0 fancycompleter==0.9.1 fastapi==0.89.1 fastavro==1.7.0 fastjsonschema==2.16.2 feast==0.26.0 filelock==3.9.0 fissix==21.11.13 flake8==6.0.0 flake8-tidy-imports==4.8.0 Flask==2.2.2 Flask-AppBuilder==4.1.4 Flask-Babel==2.0.0 Flask-Caching==2.0.1 Flask-Cors==3.0.10 Flask-JWT-Extended==4.4.4 Flask-Login==0.6.2 Flask-OpenID==1.3.0 Flask-Session==0.4.0 Flask-SQLAlchemy==2.5.1 Flask-WTF==1.0.1 flatdict==4.0.1 freezegun==1.2.2 frozenlist==1.3.3 fsspec==2022.11.0 future==0.18.2 GeoAlchemy2==0.12.5 gitdb==4.0.10 GitPython==3.1.30 google-api-core==2.11.0 google-api-python-client==2.72.0 google-auth==2.16.0 google-auth-httplib2==0.1.0 google-cloud-appengine-logging==1.2.0 google-cloud-audit-log==0.2.4 google-cloud-bigquery==3.4.1 google-cloud-bigquery-storage==2.18.0 google-cloud-core==2.3.2 google-cloud-logging==3.1.1 google-crc32c==1.5.0 google-resumable-media==2.4.0 googleapis-common-protos==1.58.0 gql==3.4.0 graphql-core==3.2.3 graphviz==0.20.1 great-expectations==0.15.41 greenlet==2.0.1 grpc-google-iam-v1==0.12.4 grpcio==1.51.1 grpcio-reflection==1.51.1 grpcio-status==1.51.1 grpcio-tools==1.51.1 gunicorn==20.1.0 h11==0.14.0 hdbcli==2.14.23 hmsclient==0.1.1 httpcore==0.16.3 httplib2==0.21.0 httptools==0.5.0 httpx==0.23.3 humanfriendly==10.0 idna==3.4 ijson==3.2.0.post0 importlib-metadata==6.0.0 importlib-resources==1.5.0 inflection==0.5.1 iniconfig==2.0.0 ipaddress==1.0.23 ipykernel==6.17.1 ipython==8.8.0 ipython-genutils==0.2.0 ipywidgets==8.0.4 iso3166==2.1.1 iso8601==1.1.0 isodate==0.6.1 isort==5.11.4 itsdangerous==2.1.2 jaraco.classes==3.2.3 jedi==0.18.2 jeepney==0.8.0 Jinja2==3.1.2 jmespath==1.0.1 JPype1==1.4.1 jsonlines==3.1.0 jsonpatch==1.32 jsonpickle==3.0.1 jsonpointer==2.3 jsonschema==4.17.3 jupyter-server==1.23.4 jupyter_client==7.4.8 jupyter_core==5.0.0 jupyterlab-pygments==0.2.2 jupyterlab-widgets==3.0.5 keyring==23.13.1 langcodes==3.3.0 lark==1.1.4 lazy-object-proxy==1.9.0 linear-tsv==1.1.0 linkify-it-py==2.0.0 lkml==1.3.1 locket==1.0.0 lockfile==0.12.2 looker-sdk==22.2.1 lxml==4.9.2 makefun==1.15.0 Mako==1.2.4 Markdown==3.4.1 markdown-it-py==2.1.0 MarkupSafe==2.1.1 marshmallow==3.19.0 marshmallow-enum==1.5.1 marshmallow-oneofschema==3.0.1 marshmallow-sqlalchemy==0.26.1 matplotlib-inline==0.1.6 maturin==0.14.2 mccabe==0.7.0 mdit-py-plugins==0.3.3 mdurl==0.1.2 mistune==2.0.4 mixpanel==4.10.0 mmh3==3.0.0 more-itertools==9.0.0 moreorless==0.4.0 moto==4.1.0 msal==1.16.0 msal-extensions==1.0.0 msrest==0.7.1 multidict==6.0.4 murmurhash==1.0.9 mypy==0.991 mypy-boto3-glue==1.26.31 mypy-boto3-s3==1.26.0.post1 mypy-boto3-sagemaker==1.26.35 mypy-boto3-sts==1.26.12 mypy-extensions==0.4.3 natsort==8.2.0 nbclassic==0.4.8 nbclient==0.6.3 nbconvert==7.2.7 nbformat==5.7.1 nest-asyncio==1.5.6 networkx==3.0 notebook==6.5.2 notebook_shim==0.2.2 ntlm-auth==1.5.0 numpy==1.24.1 oauthlib==3.2.2 okta==1.7.0 openapi-schema-validator==0.2.3 openapi-spec-validator==0.4.0 openpyxl==3.0.10 ordered-set==4.1.0 orjson==3.8.5 oscrypto==1.3.0 packaging==21.3 pandas==1.5.2 pandavro==1.5.2 pandocfilters==1.5.0 paramiko==2.12.0 parse==1.19.0 parso==0.8.3 partd==1.3.0 pathspec==0.9.0 pathy==0.10.1 pendulum==2.1.2 pep517==0.13.0 pexpect==4.8.0 phonenumbers==8.13.0 pickleshare==0.7.5 pipdeptree==2.3.3 pkginfo==1.9.6 platformdirs==2.6.2 pluggy==1.0.0 portalocker==2.6.0 preshed==3.0.8 prison==0.2.1 progressbar2==4.2.0 prometheus-client==0.15.0 prompt-toolkit==3.0.36 proto-plus==1.22.2 protobuf==4.21.12 psutil==5.9.4 psycopg2-binary==2.9.5 ptyprocess==0.7.0 pure-eval==0.2.2 pure-sasl==0.6.2 py==1.11.0 py4j==0.10.9 pyarrow==8.0.0 pyasn1==0.4.8 pyasn1-modules==0.2.8 pyathena==2.4.1 pycodestyle==2.10.0 pycountry==22.3.5 pycparser==2.21 pycryptodome==3.16.0 pycryptodomex==3.16.0 pydantic==1.10.2 pydash==5.1.2 pydeequ==1.0.1 pydruid==0.6.5 pyflakes==3.0.1 Pygments==2.14.0 PyJWT==2.6.0 pymongo==4.3.3 PyMySQL==1.0.2 PyNaCl==1.5.0 pyOpenSSL==22.0.0 pyparsing==3.0.9 pyrepl==0.9.0 pyrsistent==0.19.3 pyspark==3.0.3 pytest==7.2.0 pytest-asyncio==0.20.3 pytest-cov==4.0.0 pytest-docker==1.0.1 python-daemon==2.3.2 python-dateutil==2.8.2 python-dotenv==0.21.0 python-jose==3.3.0 python-ldap==3.4.3 python-nvd3==0.15.0 python-slugify==7.0.0 python-stdnum==1.18 python-tds==1.11.0 python-utils==3.4.5 python3-openid==3.2.0 pytz==2022.7 pytz-deprecation-shim==0.1.0.post0 pytzdata==2020.1 PyYAML==6.0 pyzmq==24.0.1 ratelimiter==1.2.0.post0 readme-renderer==37.3 redash-toolbelt==0.1.9 regex==2022.10.31 requests==2.28.1 requests-file==1.5.1 requests-mock==1.10.0 requests-ntlm==1.1.0 requests-oauthlib==1.3.1 requests-toolbelt==0.10.1 responses==0.22.0 retrying==1.3.4 rfc3986==1.5.0 rich==13.0.1 rich-click==1.5.2 rsa==4.9 ruamel.yaml==0.17.17 ruamel.yaml.clib==0.2.7 s3transfer==0.6.0 sasl3==0.2.11 schwifty==2022.9.0 scipy==1.10.0 SecretStorage==3.3.3 Send2Trash==1.8.0 setproctitle==1.3.2 simple-salesforce==1.12.2 six==1.16.0 smart-open==6.3.0 smmap==5.0.0 sniffio==1.3.0 snowflake-connector-python==2.9.0 snowflake-sqlalchemy==1.4.4 soupsieve==2.3.2.post1 spacy==3.4.3 spacy-legacy==3.0.11 spacy-loggers==1.0.4 sql-metadata==2.2.2 SQLAlchemy==1.4.41 sqlalchemy-bigquery==1.5.0 sqlalchemy-hana==0.5.0 SQLAlchemy-JSONField==1.0.1.post0 sqlalchemy-pytds==0.3.4 sqlalchemy-redshift==0.8.12 SQLAlchemy-Utils==0.39.0 sqlalchemy-vertica==0.0.5 sqlalchemy-vertica-dialect==0.1.4 sqlalchemy2-stubs==0.0.2a31 sqllineage==1.3.6 sqlparse==0.4.3 srsly==2.4.5 stack-data==0.6.2 stackprinter==0.2.10 starlette==0.22.0 swagger-ui-bundle==0.0.9 tableauserverclient==0.23.4 tableschema==1.20.2 tabulate==0.9.0 tabulator==1.53.5 tenacity==8.1.0 tensorflow-metadata==1.10.0 termcolor==2.2.0 terminado==0.17.1 text-unidecode==1.3 texttable==1.6.7 thinc==8.1.6 thrift==0.16.0 thrift-sasl==0.4.3 tinycss2==1.2.1 toml==0.10.2 tomli==2.0.1 toolz==0.12.0 tornado==6.2 tox==3.27.0 tqdm==4.64.1 traitlets==5.2.1.post0 trino==0.320.0 tuna==0.5.11 twine==4.0.2 typeguard==2.13.3 typer==0.7.0 types-awscrt==0.16.3 types-cachetools==5.2.1 types-click==0.1.12 types-click-spinner==0.1.13.1 types-dataclasses==0.6.6 types-Deprecated==1.2.9 types-freezegun==1.1.10 types-pkg-resources==0.1.3 types-protobuf==4.21.0.2 types-PyMySQL==1.0.19.1 types-pyOpenSSL==23.0.0.0 types-python-dateutil==2.8.19.5 types-pytz==2022.7.0.0 types-PyYAML==6.0.12.2 types-requests==2.28.11.7 types-s3transfer==0.6.0.post5 types-six==1.16.21.4 types-tabulate==0.9.0.0 types-termcolor==1.1.6 types-toml==0.10.8.1 types-ujson==5.7.0.0 types-urllib3==1.26.25.4 typing-inspect==0.8.0 typing_extensions==4.4.0 tzdata==2022.7 tzlocal==4.2 uc-micro-py==1.0.1 ujson==5.7.0 unicodecsv==0.14.1 uritemplate==4.1.1 urllib3==1.26.13 uvicorn==0.20.0 uvloop==0.17.0 vertica-python==1.2.0 vininfo==1.7.0 virtualenv==20.17.1 volatile==2.1.0 vulture==2.7 wasabi==0.10.1 watchfiles==0.18.1 wcmatch==8.4.1 wcwidth==0.2.5 webencodings==0.5.1 websocket-client==1.4.2 websockets==10.4 Werkzeug==2.2.2 widgetsnbextension==4.0.5 wmctrl==0.4 wrapt==1.14.1 WTForms==3.0.1 xlrd==2.0.1 xmltodict==0.13.0 yarl==1.8.2 zeep==4.2.1 zipp==3.11.0 ```Context: https://github.com/datahub-project/datahub/pull/7095
We needed to build a workaround for error handling in the OCSP cache.
The comment here provides details + repro steps: https://github.com/hsheth2/datahub/blob/d2ff513bf4b1a1fdfdd7b7b87cfdf0b8ad7d2cca/metadata-ingestion/src/datahub/ingestion/source/snowflake/snowflake_v2.py#L1578-L1598
Ideally, these errors would get caught and handled by snowflake-connector-python instead.