search

Mesnage-Org / pgfinder

Peptidoglycan MS1 Analysis Tool
https://mesnage-org.github.io/pgfinder
GNU Lesser General Public License v3.0
4 stars 2 forks source link

ci: Tests fail under Python 3.11 and 3.12 on Windows #278

Closed ns-rse closed 4 months ago

ns-rse commented 8 months ago

Describe the bug

CI fails on Python 3.11 and [3.12]() with the following...

Run pytest --cov=pgfinder
  pytest --cov=pgfinder
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
  env:
    pythonLocation: C:\hostedtoolcache\windows\Python\3.11.8\x64
    PKG_CONFIG_PATH: C:\hostedtoolcache\windows\Python\3.11.8\x64/lib/pkgconfig
    Python_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.11.8\x64
    Python[2](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:2)_ROOT_DIR: C:\hostedtoolcache\windows\Python\[3](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:3).11.8\x6[4](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:4)
    Python3_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.11.8\x64
============================= test session starts =============================
platform win32 -- Python 3.11.8, pytest-8.1.1, pluggy-1.4.0
Matplotlib: 3.8.3
Freetype: 2.6.1
rootdir: D:\a\pgfinder\pgfinder\lib
configfile: pyproject.toml
testpaths: tests
plugins: cov-4.1.0, mpl-0.17.0, regtest-2.1.1, xdist-3.[5](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:5).0
collected 19 items

tests\test_matching.py .....                                             [ 2[6](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:6)%]
tests\test_pgio.py ...FF..                                               [ 63%]
tests\test_regression.py ...                                             [ [7](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:7)[8](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:8)%]
tests\test_utils.py ...                                                  [ [9](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:9)4%]
tests\test_validation.py .                                               [[10](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:11)0%]

================================== FAILURES ===================================
_______________ test_ms_upload_reader[ftrs_test_data_5.2.ftrs] ________________

path = 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmpyzayzfrn'
onerror = <function TemporaryDirectory._rmtree.<locals>.onerror at 0x000001D3A19A3C40>

    def _rmtree_unsafe(path, onerror):
        try:
            with os.scandir(path) as scandir_it:
                entries = list(scandir_it)
        except OSError:
            onerror(os.scandir, path, sys.exc_info())
            entries = []
        for entry in entries:
            fullname = entry.path
            if _rmtree_isdir(entry):
                try:
                    if entry.is_symlink():
                        # This can only happen if someone replaces
                        # a directory with a symlink after the call to
                        # os.scandir or entry.is_dir above.
                        raise OSError("Cannot call rmtree on a symbolic link")
                except OSError:
                    onerror(os.path.islink, fullname, sys.exc_info())
                    continue
                _rmtree_unsafe(fullname, onerror)
            else:
                try:
>                   os.unlink(fullname)
E                   PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmpyzayzfrn\\ftrs_test_data.ftrs'

C:\hostedtoolcache\windows\Python\3.[11](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:12).8\x64\Lib\shutil.py:632: PermissionError

During handling of the above exception, another exception occurred:

ipywidgets_upload_output = {'content': b'SQLite format 3\x00\x10\x00\x01\x01\x00@  \x00\x02k\xbd\x00\x00\x0cp\x00\x00\x00\x00\x00\x00\x00\x00\x00...time(2022, 2, 2, 11, 6, 9, 951000, tzinfo=datetime.timezone.utc), 'name': 'ftrs_test_data.ftrs', 'size': 14274560, ...}

    def test_ms_upload_reader(ipywidgets_upload_output):
>       assert isinstance(ms_upload_reader(ipywidgets_upload_output), pd.DataFrame)

tests\test_pgio.py:26: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pgfinder\gui\internal.py:23: in ms_upload_reader
    with uploaded_file(upload) as file:
C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\contextlib.py:144: in __exit__
    next(self.gen)
pgfinder\gui\internal.py:29: in uploaded_file
    with tempfile.TemporaryDirectory() as tempdir:
C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\tempfile.py:943: in __exit__
    self.cleanup()
C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\tempfile.py:947: in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\tempfile.py:929: in _rmtree
    _shutil.rmtree(name, onerror=onerror)
C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\shutil.py:787: in rmtree
    return _rmtree_unsafe(path, onerror)
C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\shutil.py:634: in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

func = <built-in function unlink>
path = 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmpyzayzfrn\\ftrs_test_data.ftrs'
exc_info = (<class 'PermissionError'>, PermissionError([13](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:14), 'The process cannot access the file because it is being used by another process'), <traceback object at 0x000001D3A1967FC0>)

    def onerror(func, path, exc_info):
        if issubclass(exc_info[0], PermissionError):
            if repeated and path == name:
                if ignore_errors:
                    return
                raise

            try:
                if path != name:
                    _resetperms(_os.path.dirname(path))
                _resetperms(path)

                try:
>                   _os.unlink(path)
E                   PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmpyzayzfrn\\ftrs_test_data.ftrs'

C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\tempfile.py:893: PermissionError
_______________ test_ms_upload_reader[ftrs_test_data_3.11.ftrs] _______________

path = 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmp555lx39l'
onerror = <function TemporaryDirectory._rmtree.<locals>.onerror at 0x000001D3A19B0900>

    def _rmtree_unsafe(path, onerror):
        try:
            with os.scandir(path) as scandir_it:
                entries = list(scandir_it)
        except OSError:
            onerror(os.scandir, path, sys.exc_info())
            entries = []
        for entry in entries:
            fullname = entry.path
            if _rmtree_isdir(entry):
                try:
                    if entry.is_symlink():
                        # This can only happen if someone replaces
                        # a directory with a symlink after the call to
                        # os.scandir or entry.is_dir above.
                        raise OSError("Cannot call rmtree on a symbolic link")
                except OSError:
                    onerror(os.path.islink, fullname, sys.exc_info())
                    continue
                _rmtree_unsafe(fullname, onerror)
            else:
                try:
>                   os.unlink(fullname)
E                   PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmp555lx39l\\ftrs_test_data.ftrs'

C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\shutil.py:632: PermissionError

During handling of the above exception, another exception occurred:

ipywidgets_upload_output = {'content': b'SQLite format 3\x00\x10\x00\x01\x01\x00@  \x00\x00\x00\x17\x00\x00\r\x9d\x00\x00\x00\x00\x00\x00\x00\x00...time(2022, 2, 2, 11, 6, 9, 951000, tzinfo=datetime.timezone.utc), 'name': 'ftrs_test_data.ftrs', 'size': [14](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:15)274560, ...}

    def test_ms_upload_reader(ipywidgets_upload_output):
>       assert isinstance(ms_upload_reader(ipywidgets_upload_output), pd.DataFrame)

tests\test_pgio.py:26: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pgfinder\gui\internal.py:23: in ms_upload_reader
    with uploaded_file(upload) as file:
C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\contextlib.py:144: in __exit__
    next(self.gen)
pgfinder\gui\internal.py:29: in uploaded_file
    with tempfile.TemporaryDirectory() as tempdir:
C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\tempfile.py:943: in __exit__
    self.cleanup()
C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\tempfile.py:947: in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\tempfile.py:929: in _rmtree
    _shutil.rmtree(name, onerror=onerror)
C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\shutil.py:787: in rmtree
    return _rmtree_unsafe(path, onerror)
C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\shutil.py:634: in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

func = <built-in function unlink>
path = 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmp555lx39l\\ftrs_test_data.ftrs'
exc_info = (<class 'PermissionError'>, PermissionError(13, 'The process cannot access the file because it is being used by another process'), <traceback object at 0x000001D3A[19](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:20)6BC80>)

    def onerror(func, path, exc_info):
        if issubclass(exc_info[0], PermissionError):
            if repeated and path == name:
                if ignore_errors:
                    return
                raise

            try:
                if path != name:
                    _resetperms(_os.path.dirname(path))
                _resetperms(path)

                try:
>                   _os.unlink(path)
E                   PermissionError: [WinError [32](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:33)] The process cannot access the file because it is being used by another process: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmp555lx[39](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:40)l\\ftrs_test_data.ftrs'

C:\hostedtoolcache\windows\Python\3.11.8\x[64](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:65)\Lib\tempfile.py:[89](https://github.com/Mesnage-Org/pgfinder/actions/runs/8365887728/job/22904806059#step:5:90)3: PermissionError
---------------------------- pytest-regtest report ----------------------------

Desktop (please complete the following information):

Additional context

First cropped up in #269

Would be worth checking the minor Python 3.11.* version on runs prior that passed if the logs are still around.

TheLostLambda commented 4 months ago

Also fixed in the latest PR!