fsspec / universal_pathlib

pathlib api extended to use fsspec backends
MIT License
251 stars 44 forks source link

Webdav tests sporadically error on windows #106

Closed ap-- closed 1 year ago

ap-- commented 1 year ago

They fail in the webdav_fixture. Rerunning the job usually recovers it. We need to make some improvements here to be more reliable on windows.


@pytest.fixture
    def webdav_fixture(local_testdir, webdav_server):
        webdav_path, webdav_url = webdav_server
        if os.path.isdir(webdav_path):
            shutil.rmtree(webdav_path, ignore_errors=True)
        try:
>           shutil.copytree(local_testdir, webdav_path)

upath\tests\conftest.py:339: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\shutil.py:557: in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\shutil.py:[45](https://github.com/fsspec/universal_pathlib/actions/runs/5304477693/jobs/9600822124?pr=105#step:5:46)8: in _copytree
    os.makedirs(dst, exist_ok=dirs_exist_ok)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

name = 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-0\\webdav0'
mode = 511, exist_ok = False

    def makedirs(name, mode=0o777, exist_ok=False):
        """makedirs(name [, mode=0o777][, exist_ok=False])

        Super-mkdir; create a leaf directory and all intermediate ones.  Works like
        mkdir, except that any intermediate path segment (not just the rightmost)
        will be created if it does not exist. If the target directory already
        exists, raise an OSError if exist_ok is False. Otherwise no exception is
        raised.  This is recursive.

        """
        head, tail = path.split(name)
        if not tail:
            head, tail = path.split(head)
        if head and tail and not path.exists(head):
            try:
                makedirs(head, exist_ok=exist_ok)
            except FileExistsError:
                # Defeats race condition when another thread created the path
                pass
            cdir = curdir
            if isinstance(tail, bytes):
                cdir = bytes(curdir, 'ASCII')
            if tail == cdir:           # xxx/newdir/. exists if xxx/newdir exists
                return
        try:
>           mkdir(name, mode)
E           FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-0\\webdav0'

C:\hostedtoolcache\windows\Python\3.8.10\x[64](https://github.com/fsspec/universal_pathlib/actions/runs/5304477693/jobs/9600822124?pr=105#step:5:65)\lib\os.py:223: FileExistsError
ap-- commented 1 year ago

I assume #103 was just hiding this error for a while, and was the wrong way to address this.

ap-- commented 1 year ago

😞 seems it didn't do the trick...