Open bennwei opened 3 months ago
I am having the same issue. It looks like airbyte-cdk (https://github.com/tcboles/airbyte/blob/master/airbyte-cdk/python/pyproject.toml) and a few connectors would need to be updated to use the latest pendulum version to make this work with projects that already use the latest pendulum version or have dependencies that do.
I am also facing the same error.
I had tried many ways in my local env and can not bypass the pendulum==2.1.2 issue. But when I tried a colab demo example and airbyte was able to installed without issue: https://colab.research.google.com/github/airbytehq/quickstarts/blob/master/pyairbyte_notebooks/PyAirbyte_Basic_Features_Demo.ipynb
This happens because distutils was deprecated in Python 3.12. The dependency seems to be fixed in Pendulum 3.0, so bumping the dependency should do the trick.
@notpushkin How did you resolve this on your side? I tried to bump Pendulum==3.0, but the issues is on the SDK airbyte-cdk (https://github.com/tcboles/airbyte/blob/master/airbyte-cdk/python/pyproject.toml) as @tcboles mentioned above that SDK is not compatible with Pendulum==3.0 install. Let me know if any solution you can bypass this issue when there is no package updates from the author.
@notpushkin, @neerajgupta2407, @bennwei, @tcboles - Thanks everyone for reporting this and sharing your observations here.
It sounds like this issue is specific to Python 3.12+, where distutils
is no longer available, while at the same time being required by Pendulum 2.x
. As others have noted, the CDK constrains Pendulum to <3.0
so I think there are just two solutions:
distutils
is still available.Does that feel like an accurate summary? If so, I'll open an issue shortly to request a Pendulum version bump in the CDK upstream.
I will look into breaking changes... UPDATE: I have looked into breaking changes in Pendulum 3.0 and one change that I was able to identify as impactful to some connectors is the renaming of the Period
class to Interval
. Some of our connectors use this type and it would be non-trivial to make them forwards and backwards compatible across major versions of Pendulum. I'm curious if there are any other workarounds 2.x to make Pendulum install correctly. I'll look deeper into this and report back...
Please feel free to chime in if this sounds like an appropriate path forward or if you have any other ideas or feedback. Thanks!
A very highly esteemed ex-colleague of mine @edgarrmondragon posted this comment:
Pendulum 3.0.0b1 has wheels for M1 macs and Python 3.12, but builds of Pendulum 2.* on Python 3.12+ seem like they'll be forever broken.
In terms of upgrading, I worry about supporting only Pendulum 3.x, since this will break side-by-side installations if users are using libraries that might still be on Pendulum 2.x. For a while, at least, it seems the safest option is to downgrade to Python 3.11 if possible. The longer-term option seem to be (1) update the CDK and PyAirbyte to allow >=2.0,<4.0
, and then (2) update the connectors to use the more restrictive, but forward-looking >=3.0,<4.0
.
@bennwei
How did you resolve this on your side?
I didn't (yet). I was trying to play around with PyAirbyte to see if it's a good fit for a project I'm working on, so when it didn't work I just moved on ¯\_(ツ)_/¯
@aaronsteers Thanks for the support.
Now I am using python version 3.10 and it's working fine.
I tried to install PyAirbyte, but installation failed due to pendulum==2.1.2:
Also failed trying using pip install: