danilop / yas3fs

YAS3FS (Yet Another S3-backed File System) is a Filesystem in Userspace (FUSE) interface to Amazon S3. It was inspired by s3fs but rewritten from scratch to implement a distributed cache synchronized by Amazon SNS notifications. A web console is provided to easily monitor the nodes of a cluster.
http://danilop.github.io/yas3fs
MIT License
644 stars 98 forks source link

Undocumented requests package version requirement #196

Closed mojibake-umd closed 2 years ago

mojibake-umd commented 2 years ago

I will try to make a more detailed issue later.

Summary. A newer version of urllib3 than is provided by RHEL7 yum package repos is required. yum version 1.10.2 don't know when breakage introduced, but at least latest urllib3 package via pip works 1.26.10

When I say works...If urllib3 is too old the yas3fs service will fail to start with error like

Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: Traceback (most recent call last): Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: File "/bin/yas3fs", line 5, in Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: from pkg_resources import load_entry_point Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: File "/usr/lib/python2.7/site-packages/pkg_resources/init.py", line 3251, in <modul Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: @_call_aside Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: File "/usr/lib/python2.7/site-packages/pkg_resources/init.py", line 3235, in call Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: f(*args, **kwargs) Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: File "/usr/lib/python2.7/site-packages/pkg_resources/init.py", line 3264, in _initi Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: working_set = WorkingSet._build_master() Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: File "/usr/lib/python2.7/site-packages/pkg_resources/init.py", line 585, in build Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: return cls._build_from_requirements(requires) Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: File "/usr/lib/python2.7/site-packages/pkg_resources/init.py", line 598, in build Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: dists = ws.resolve(reqs, Environment()) Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: File "/usr/lib/python2.7/site-packages/pkg_resources/init.py", line 786, in resolve Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: raise DistributionNotFound(req, requirers) Jul 18 13:57:02 sftp.it-eng-webhosting.aws.umd.edu env[12600]: pkg_resources.DistributionNotFound: The 'urllib3<1.27,>=1.25.4' distribution was not fo

Based on error message with botocore 1.20.104 installed, It looks like it wants at least urllib3 1.25.4 or above. Or maybe need to limit botocore version installed to avoid breakage.

I think the changelog information in https://botocore.amazonaws.com/v1/documentation/api/latest/index.html indicates that botocore >=1.11.0 is what is breaking vendored urllib3 and requests.

mojibake-umd commented 2 years ago

Actually, in my case a mismatch of vendor (RHEL yum) provided pip modules and pip provided modules were causing issue. Uninstalling pip provided boto, botocore, urllib3, requests and reinstalling they via yum worked.

jazzl0ver commented 2 years ago

There's a special warning in README regarding the pip installation:

PIP installation is no longer supported. Use "git clone" instead.