Closed jmahlik closed 1 year ago
Thank you @jmahlik, I agree with your points. PR #1992 should address this as part of release 3.0
Thank you @jmahlik, I agree with your points. PR #1992 should address this as part of release 3.0
Awesome, just ran our internal test suites off the release-3.0.0
branch and this completely resolves all the issues and passes without breakage :). Even on python 3.11.
Hi @jaidisido, this optional dependency is the last blocker for Python 3.11 support. When will this be available on PyPI as a 3.0.0rc release?
@menaitm I did the workaround from https://github.com/aws/aws-sdk-pandas/issues/1714#issuecomment-1435156127 on 2.x and it worked wonderfully. Pin gremlinpython==3.6.3rc1
, assuming you aren't actually using it.
Is your idea related to a problem? Please describe. There are quite a few issues related to dependencies with the current installation setup. In our organization, the dependency conflicts/issues are getting to the point of considering removing aws-sdk-pandas as a dependency and re-implementing some of the functionality.
I (and likely others) am hoping this situation can be improved upon, because aws-sdk-pandas IS an awesome library! Happy to help make this happen.
Main issues
Describe the solution you'd like With an upcoming 3.0 release, I wonder if it would be possible to comb though the dependencies and move the specific connector/service related deps to a service-related extra. i.e everything for redshift goes in a "redshift" extra, everything for lakeformation goes in a "lakeformation" extra etc. A major release would be the perfect timing for this IMO.
Dask has a pretty good setup/example of how to handle this and educate users about extras. Maybe consider doing something similar for aws-sdk-pandas?
https://docs.dask.org/en/stable/install.html#pip
Additionally, asking for the dependency restrictions to be relaxed so users can incorporate bug fixes. Mostly around pandas, pyarrow, boto and numpy. It should be up to the user to pin and or upgrade. If downstream isn't following a best practice, educate them on how to.
Transitives
On a fresh venv I end up with the following 52 packages from installing awswrangler with no extras. For comparison, a venv with the "core" requirements (pandas, numpy pyarrow, boto3) installed there are 16 packages. All of these are not needed for interacting with an individual service.
pip list
Output from pipdeptree.