search

snowflakedb / snowflake-connector-python

Snowflake Connector for Python
https://pypi.python.org/pypi/snowflake-connector-python/
Apache License 2.0
585 stars 468 forks source link

SNOW-1460707: Erroneously Looking for Config.toml File when using DATABASES dictionary in settings.py #1959

Closed dnich02f closed 3 months ago

dnich02f commented 4 months ago

Python version

Python 3.8

Operating system and processor architecture

Linux-6.6.16-linuxkit-aarch64-with-glibc2.36

Installed packages

Django==4.2.11
django-snowflake==4.2.1
gunicorn==20.0.0

What did you do?

{"status": "error", "message": "[Errno 13] Permission denied: '/root/.config/snowflake/config.toml'"}

I am using django-snowflake, which uses this package.

It is my understanding that since I am referencing all of my environment variables from the DATABASES dictionary (using os.getenv), that the config.toml is not necessary. So I was confused as to why all of a sudden this was file was being searched for. And to be clear, it's not just that Permission denied, it is that the file and directory did not exist at all.

How I fixed the issue:
I added snowfalke-connector-python==3.8.1 to my requirements.txt file before django-snowflake==4.2.*

Why?
I found this in the dist-info/METADATA for snowflake-connector-python 3.10.1:

v3.9.0(April 20,2024)
Added easy logging configuration so that users can easily generate log file by setup log config in $SNOWFLAKE_HOME/config.toml.
Improved s3 acceleration logic when connecting to China endpoint.

Not sure how contributors want to address it, or if it just might be useful for someone searching for solutions to find this.

What did you expect to see?

Nothing. I expected to not have this error at all, as I wasn't experiencing it before.

Can you set logging to DEBUG and collect the logs?

Traceback (most recent call last):
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapper_view
    return view_func(request, *args, **kwargs)
  File "xxx", line xxx, in some_function
    snowflake_object = django ORM reference,
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/db/models/query.py", line 1057, in first
    for obj in queryset[:1]:
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/db/models/query.py", line 398, in __iter__
    self._fetch_all()
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/db/models/query.py", line 1881, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1560, in execute_sql
    cursor = self.connection.cursor()
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/db/backends/base/base.py", line 330, in cursor
    return self._cursor()
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/db/backends/base/base.py", line 306, in _cursor
    self.ensure_connection()
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/db/backends/base/base.py", line 289, in ensure_connection
    self.connect()
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/db/backends/base/base.py", line 270, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/django_snowflake/base.py", line 137, in get_new_connection
    return Database.connect(**conn_params)
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/snowflake/connector/__init__.py", line 55, in Connect
    return SnowflakeConnection(**kwargs)
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/snowflake/connector/connection.py", line 388, in __init__
    easy_logging = EasyLoggingConfigPython()
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/snowflake/connector/log_configuration.py", line 24, in __init__
    self.parse_config_file()
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/snowflake/connector/log_configuration.py", line 27, in parse_config_file
    CONFIG_MANAGER.read_config()
  File "/home/ubuntu/app/venv/lib/python3.8/site-packages/snowflake/connector/config_manager.py", line 323, in read_config
    if not filep.exists():
  File "/usr/lib/python3.8/pathlib.py", line 1407, in exists
    self.stat()
  File "/usr/lib/python3.8/pathlib.py", line 1198, in stat
    return self._accessor.stat(self)
PermissionError: [Errno 13] Permission denied: '/root/.config/snowflake/config.toml'
sfc-gh-dszmolka commented 4 months ago

hi and thank you for reporting this, also for sharing your workaround. Indeed in django-snowflake with v4.2, we seem to have bumped the snowflake-connector-python minimum dependency from 2.7.4 to 3.6.0 which will install the latest one.

As you noticed, in the more recent PythonConnector versions we now try to read the config from a config.toml but we can agree that perhaps the non-existence of this non-mandatory config file could be handled differently. We'll take a look

edit: looks similar to https://github.com/snowflakedb/snowflake-connector-python/issues/1943

dnich02f commented 4 months ago

@sfc-gh-dszmolka , thank you for looking into it. I don't have a strong opinion on handling the config file, necessarily. Just wanted to bring it to your attention. Thanks again!

edit: I agree that it is similar/the same as the issue you mentioned.

sfc-gh-dszmolka commented 4 months ago

Fix PR already seems to be raised: https://github.com/snowflakedb/snowflake-connector-python/pull/1965

sfc-gh-dszmolka commented 4 months ago

reopening to be able to keep track with the release and will close once the release having this fix, is out

sfc-gh-dszmolka commented 3 months ago

this should have released with v3.11.0 now