search

inveniosoftware / invenio-files-rest

REST API for uploading/downloading files for Invenio.
https://invenio-files-rest.readthedocs.io
MIT License
9 stars 67 forks source link

fs>=0.5.4,<2.0 with setuptools>=58.0.0 #264

Open thlava-cesnet opened 3 years ago

thlava-cesnet commented 3 years ago

Description

We have caught an issue, which seems to be related with an invenio_files_rest's dependency on old python v2 code in "fs==0.5.4" package (as result of 'fs>=0.5.4,<2.0' constraints) and building with new versions of setuptools>=58.0.0, where have been removed "2to3" support during builds.

Suggested solution

Although there is workaround using setptools<58.0.0, better solution may be in switch to newer version of "fs" package. I know it could be tricky ;) (https://github.com/inveniosoftware/invenio-files-rest/search?q=fs&type=issues) but nevertheless: could be generally possible to remove "fs<2.0" constraint without big bad side-effects?

Alternative solution

Using setuptools<58.0.0 for build.

Additional context

original error:

...
/usr/local/lib/python3.8/site-packages/invenio_files_rest/storage/__init__.py:14: in <module>
    from .pyfs import PyFSFileStorage, pyfs_storage_factory
/usr/local/lib/python3.8/site-packages/invenio_files_rest/storage/pyfs.py:14: in <module>
    from fs.opener import opener
/usr/local/lib/python3.8/site-packages/fs/__init__.py:22: in <module>
    from fs import errors
E     File "/usr/local/lib/python3.8/site-packages/fs/errors.py", line 220
E       except ResourceNotFoundError, e:
E                                   ^
E   SyntaxError: invalid syntax
...

setuptools history: (https://setuptools.readthedocs.io/en/latest/history.html)

...
v58.0.0
  Breaking Changes
    #2086: Removed support for 2to3 during builds. Projects should port to a unified codebase or pin to an older version of Setuptools using PEP 518 build-requires.
...

invenio-files-rest/setup.py: (https://github.com/inveniosoftware/invenio-files-rest/blob/27e067be62c0b76e96428397cae84555b016f614/setup.py#L59)

lnielsen commented 3 years ago

Yep, basically almost all of the CI builds are failing now because of this issue. We're looking into solutions, but unfortunately it's a rather difficult upgrade to remove the constraint - especially because of xrootdpyfs.

Sinclert commented 2 years ago

+1. This problem affects one of our Pipenv managed projects at NYU.

Update:

Current work-around to install a pre-58 version of setuptools with Pipenv:

pipenv run python3 -m pip install 'setuptools==57.5.0'
pipenv install
lnielsen commented 2 years ago

Since a couple of days there's new docker images out with setuptools>58.0. @max-moser is working on a fix for Invenio-Files-REST, but it's a bigger task so will take a bit of time to push out the releases.