search

inveniosoftware / invenio-files-rest

REST API for uploading/downloading files for Invenio.
https://invenio-files-rest.readthedocs.io
MIT License
9 stars 66 forks source link

Latest version of fs(2.0.0a1) breaks the compatibility #130

Open spirosdelviniotis opened 7 years ago

spirosdelviniotis commented 7 years ago

After running inspire-next on travis-ci (https://travis-ci.org/inspirehep/inspire-next/jobs/170102212) we found that the latest fs module version(2.0.0a1) breaks the api that this module uses.

For example there is no opener but Opener and some of the errors were renamed, for example DirectoryNotEmptyError to DirectoryNotEmpty.

jirikuncar commented 7 years ago

fs>=2.0.0a1 looks like rewrite from scratch: https://github.com/PyFilesystem/pyfilesystem2 We should probably pin the dependency until we figure out the status.

david-caro commented 6 years ago

@lnielsen I had a quick look at this, it seems that at least one functionality we are depending on has changed, there's no way to remove the root directory of a fs instance anymore: https://github.com/PyFilesystem/pyfilesystem2/blob/7e5d556b5c760a0d61ec679cf80081189053b4d7/fs/osfs.py#L325

and we are doing it here: https://github.com/inveniosoftware/invenio-files-rest/blob/master/invenio_files_rest/storage/pyfs.py#L85

There's no such check for other types of filesystems, like eos for example: https://github.com/inveniosoftware/xrootdpyfs/blob/master/xrootdpyfs/fs.py#L319

Should it then check specifically if it's OSFS and try to remove it with os directly in that case? It clearly looks like the intention is to avoid removing the roof of the filesystem, maybe we should not try to do that then? But that might create lots of empty dirs (as we create them with the create flag when creating the filesystem, but can't remove them after...). wdyt?

lnielsen commented 6 years ago

I don't know if v0.5 and v2 can be install at the same time - if they both can be installed, we might just do a class PyFS2FileStorage(PyFSFileStorage) class that fixes this specific isssue we have.

My main concern for upgrading to pyfilesystem2 is however inveniosoftware/xrootdpyfs which likely have quite a lot of fixes.