search

PyFilesystem / pyfilesystem

Python filesystem abstraction layer
http://pyfilesystem.org/
BSD 3-Clause "New" or "Revised" License
287 stars 63 forks source link

fs.utils.copydir: RuntimeError: maximum recursion depth exceeded in cmp #251

Open zopyx opened 8 years ago

zopyx commented 8 years ago

The following script gives me the attached error

from fs.contrib.davfs import DAVFS
from fs.osfs import OSFS
import fs.utils

source = OSFS('/tmp/bookalope')
target = DAVFS('http://localhost:6080/exist/webdav/db', credentials=dict(username='admin', password='onkopedia'))

print source.listdir()
print target.listdir()

fs.utils.copydir((source, 'zopyx-nimbudocs'), (target, '.'))

  File "/home/ajung/.buildout/eggs/fs-0.5.4-py2.7.egg/fs/contrib/davfs/__init__.py", line 556, in makedir
    self.makedir(dirname(path),recursive=True,allow_recreate=True)
  File "/home/ajung/.buildout/eggs/fs-0.5.4-py2.7.egg/fs/contrib/davfs/__init__.py", line 556, in makedir
    self.makedir(dirname(path),recursive=True,allow_recreate=True)
  File "/home/ajung/.buildout/eggs/fs-0.5.4-py2.7.egg/fs/contrib/davfs/__init__.py", line 556, in makedir
    self.makedir(dirname(path),recursive=True,allow_recreate=True)
  File "/home/ajung/.buildout/eggs/fs-0.5.4-py2.7.egg/fs/contrib/davfs/__init__.py", line 556, in makedir
    self.makedir(dirname(path),recursive=True,allow_recreate=True)
  File "/home/ajung/.buildout/eggs/fs-0.5.4-py2.7.egg/fs/contrib/davfs/__init__.py", line 556, in makedir
    self.makedir(dirname(path),recursive=True,allow_recreate=True)
  File "/home/ajung/.buildout/eggs/fs-0.5.4-py2.7.egg/fs/contrib/davfs/__init__.py", line 556, in makedir
    self.makedir(dirname(path),recursive=True,allow_recreate=True)
  File "/home/ajung/.buildout/eggs/fs-0.5.4-py2.7.egg/fs/contrib/davfs/__init__.py", line 556, in makedir
    self.makedir(dirname(path),recursive=True,allow_recreate=True)
  File "/home/ajung/.buildout/eggs/fs-0.5.4-py2.7.egg/fs/contrib/davfs/__init__.py", line 556, in makedir
    self.makedir(dirname(path),recursive=True,allow_recreate=True)
  File "/home/ajung/.buildout/eggs/fs-0.5.4-py2.7.egg/fs/contrib/davfs/__init__.py", line 556, in makedir
    self.makedir(dirname(path),recursive=True,allow_recreate=True)
  File "/home/ajung/.buildout/eggs/fs-0.5.4-py2.7.egg/fs/contrib/davfs/__init__.py", line 556, in makedir
    self.makedir(dirname(path),recursive=True,allow_recreate=True)
  File "/home/ajung/.buildout/eggs/fs-0.5.4-py2.7.egg/fs/contrib/davfs/__init__.py", line 549, in makedir
    response = self._request(path,"MKCOL")
  File "/home/ajung/.buildout/eggs/fs-0.5.4-py2.7.egg/fs/contrib/davfs/__init__.py", line 260, in _request
    resp = self._raw_request(url,method,body,headers)
  File "/home/ajung/.buildout/eggs/fs-0.5.4-py2.7.egg/fs/contrib/davfs/__init__.py", line 321, in _raw_request
    resp = con.getresponse()
  File "/opt/buildout.python/parts/opt/lib/python2.7/httplib.py", line 1136, in getresponse
    response.begin()
  File "/opt/buildout.python/parts/opt/lib/python2.7/httplib.py", line 485, in begin
    self.msg = HTTPMessage(self.fp, 0)
  File "/opt/buildout.python/parts/opt/lib/python2.7/mimetools.py", line 25, in __init__
    rfc822.Message.__init__(self, fp, seekable)
  File "/opt/buildout.python/parts/opt/lib/python2.7/rfc822.py", line 108, in __init__
    self.readheaders()
  File "/opt/buildout.python/parts/opt/lib/python2.7/httplib.py", line 340, in readheaders
    elif self.islast(line):
  File "/opt/buildout.python/parts/opt/lib/python2.7/rfc822.py", line 222, in islast
    return line in _blanklines
RuntimeError: maximum recursion depth exceeded in cmp
lurch commented 8 years ago

If you're exceeding the recursion depth, then my initial guess would be that your filesystem is too deeply nested?

Can you upload the output of the following script somewhere?

from fs.osfs import OSFS
source = OSFS('/tmp/bookalope')
source.tree(max_levels=None)

EDIT: Added the max_levels parameter, otherwise tree stops at 5 levels deep

zopyx commented 8 years ago

The really nested. The folder just contains a dozen folders.

lurch commented 8 years ago

I've been having a play around today, and eventually managed to reproduce this; and you're right it isn't a nesting problem ;-)

I'm not sure exactly where the fault lies (I've never done any WebDAV stuff before, so it could be with either the fs.contrib.davfs code, or it could be with the eXist-db WebDAV implementation). But if you try to do target.makedir('.') then you get the error fs.errors.ParentDirectoryMissingError: Parent directory is missing: .. This means the code inside https://github.com/PyFilesystem/pyfilesystem/blob/master/fs/contrib/davfs/__init__.py#L548 then tries to do target.makedir('') which also gives a ParentDirectoryMissingError, and so the code gets stuck in an inifinite loop, until it reaches the recursion limit.

So a simple workaround for your test-code above would be to change it to:

fs.utils.copydir((source, 'zopyx-nimbudocs'), target)

which seems to bypass the problem.