aws / amazon-redshift-python-driver

Redshift Python Connector. It supports Python Database API Specification v2.0.
Apache License 2.0
205 stars 75 forks source link

Using deprecated pkg_resources instead of importlib #127

Closed bneijt closed 2 years ago

bneijt commented 2 years ago

Driver version

2.0.908

Redshift version

none, used via aws wrangler

Client Operating System

Linux

Python version

Python 3.9.12

Table schema

none

Problem description

Setuptools pkg_resources has been deprecated in favor of importlib. I would expect the code to work without pkg_resources, for example by using a compatibility import statement to fall-back to pkg_resources if required or using importlib_resources to support older versions of python.

  1. Expected behaviour: Use importlib_resources for older versions of Python and nolonger depend on pkg_resources.
  2. Actual behaviour: Uses pkg_resources directly and fails
  3. Error message/stack trace:
    
    ModuleNotFoundError                       Traceback (most recent call last)
    Input In [1], in <cell line: 2>()
      1 import pandas as pd
    ----> 2 import awswrangler as wr
      3 from loguru import logger
      4 import os

File ~/program/tmobile/bna-batch/.venv/lib/python3.9/site-packages/awswrangler/init.py:10, in 1 """Initial Module. 2 3 Source repository: https://github.com/awslabs/aws-data-wrangler 4 Documentation: https://aws-data-wrangler.readthedocs.io/ 5 6 """ 8 import logging as _logging ---> 10 from awswrangler import ( # noqa 11 athena, 12 catalog, 13 chime, 14 cloudwatch, 15 data_api, 16 dynamodb, 17 emr, 18 exceptions, 19 lakeformation, 20 mysql, 21 neptune, 22 opensearch, 23 oracle, 24 postgresql, 25 quicksight, 26 redshift, 27 s3, 28 secretsmanager, 29 sqlserver, 30 sts, 31 timestream, 32 ) 33 from awswrangler.metadata import description, license, title, version # noqa 34 from awswrangler._config import config # noqa

File ~/program/tmobile/bna-batch/.venv/lib/python3.9/site-packages/awswrangler/redshift.py:13, in 11 import pandas as pd 12 import pyarrow as pa ---> 13 import redshift_connector 15 from awswrangler import _data_types 16 from awswrangler import _databases as _db_utils

File ~/program/tmobile/bna-batch/.venv/lib/python3.9/site-packages/redshift_connector/init.py:4, in 1 import logging 2 import typing ----> 4 from redshift_connector import plugin 5 from redshift_connector.config import DEFAULT_PROTOCOL_VERSION, ClientProtocolVersion 6 from redshift_connector.core import BINARY, Connection, Cursor

File ~/program/tmobile/bna-batch/.venv/lib/python3.9/site-packages/redshift_connector/plugin/init.py:4, in 2 from .azure_credentials_provider import AzureCredentialsProvider 3 from .browser_azure_credentials_provider import BrowserAzureCredentialsProvider ----> 4 from .browser_azure_oauth2_credentials_provider import ( 5 BrowserAzureOAuth2CredentialsProvider, 6 ) 7 from .browser_saml_credentials_provider import BrowserSamlCredentialsProvider 8 from .idp_credentials_provider import IdpCredentialsProvider

File ~/program/tmobile/bna-batch/.venv/lib/python3.9/site-packages/redshift_connector/plugin/browser_azure_oauth2_credentials_provider.py:8, in 6 from redshift_connector.error import InterfaceError 7 from redshift_connector.plugin.credential_provider_constants import azure_headers ----> 8 from redshift_connector.plugin.jwt_credentials_provider import JwtCredentialsProvider 9 from redshift_connector.redshift_property import RedshiftProperty 11 if typing.TYPE_CHECKING:

File ~/program/tmobile/bna-batch/.venv/lib/python3.9/site-packages/redshift_connector/plugin/jwt_credentials_provider.py:6, in 3 from abc import abstractmethod 5 from redshift_connector.error import InterfaceError ----> 6 from redshift_connector.iam_helper import IamHelper 7 from redshift_connector.plugin.i_native_plugin import INativePlugin 8 from redshift_connector.plugin.idp_credentials_provider import IdpCredentialsProvider

File ~/program/tmobile/bna-batch/.venv/lib/python3.9/site-packages/redshift_connector/iam_helper.py:6, in 3 import logging 4 import typing ----> 6 import pkg_resources 7 from dateutil.tz import tzutc 8 from packaging.version import Version

ModuleNotFoundError: No module named 'pkg_resources'


5. Any other details that can be helpful:
 - If this change can be released in a minor patch, [aws-skd-pandas](https://github.com/aws/aws-sdk-pandas/blob/main/pyproject.toml#L35) should pick it up automatically.
 - This is closely related to issue #122 

## Python Driver trace logs
None

## Reproduction code
- Create a venv with python 3.9
- Drop setuptools from the venv
- Try to import the driver
Brooke-white commented 2 years ago

Hey @bneijt , thanks for the heads up. we will resolve this in the next release :)

Brooke-white commented 2 years ago

2.0.909 has been released with this fix.