[ADAP-789] [Feature] Upgrade PyHive to support Python 3.11

[JCZuurmond]

Is this your first time submitting a feature request?

• [X] I have read the expectations for open source contributors
• [X] I have searched the existing issues, and I could not find an existing issue for this feature
• [X] I am requesting a straightforward extension of existing dbt-spark functionality, rather than a Big Idea better suited to a discussion

Describe the feature

It looks like PyHive support Python 3.11 now. Let's investigate if we can use, so that dbt-spark can support Python 3.11.

Describe alternatives you've considered

N.a.

Who will this benefit?

Those who use Python 3.11

Are you interested in contributing this feature?

yes, other contributions are also welcome

Anything else?

No response

[shishircc]

Thrift is only way I know dbt can work with EMR. Let me know if I am missing something here. The Thriftserver support would be crucial.

[leo-schick]

The issue here is related to the sasl package which does not work with python 3.11 anymore. To make this work, you need to install pyhive with extra hive_pure_sasl which uses pure-sasl instead of the sasl package. To make this work, dbt-spark should use pyhive[hive_pure_sasl] instead of just pyhive when installing dbt-spark[pyhive].

You can easily reproduce this issue by running pip install pyhive vs pip install pyhive[hive_pure_sasl] on a python 3.11 installation.

[dbeatty10]

@JCZuurmond were you able to find out if @leo-schick suggestion works or not?

https://github.com/dbt-labs/dbt-core/issues/9491 is currently blocked on this.

[JCZuurmond]

@dbeatty10 : I did not have time to verify this. Could you verify it and change the code accordingly?

[dbeatty10]

@JCZuurmond good news: looks like this was resolved in https://github.com/dbt-labs/dbt-spark/pull/818

https://github.com/dbt-labs/dbt-spark/blob/613fa58ff9d1f06877ad8790a145d91f5913f862/setup.py#L48 https://github.com/dbt-labs/dbt-spark/blob/613fa58ff9d1f06877ad8790a145d91f5913f862/requirements.txt#L1