tarpas / pytest-testmon

Selects tests affected by changed files. Executes the right tests first. Continuous test runner when used with pytest-watch.
https://testmon.org
MIT License
800 stars 54 forks source link

Windows: Collection fails and breaks DB #169

Closed robertschweizer closed 7 months ago

robertschweizer commented 2 years ago

On Windows, when I have source and test files in separate subfolders, testmon does not rerun tests correctly after I changed files. If I run it again, testmon creates a completely new DB file.

Console output:

pytest --testmon
============================= test session starts =============================
platform win32 -- Python 3.8.10, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
testmon: new DB, environment: default
rootdir: C:\Users\kftf964\playground\testmon\test_dir
plugins: testmon-1.2.0
collected 1 item

tests\test_script.py .                                                   [100%]

============================== 1 passed in 0.14s ==============================

# Change code
printf "def fun():\n    print('b')" > source/script.py

pytest --testmon || echo "non-zero exit-code"
============================= test session starts =============================
platform win32 -- Python 3.8.10, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
testmon: changed files: tests/test_script.py, skipping collection of 2 files, environment: default
rootdir: C:\Users\kftf964\playground\testmon\test_dir
plugins: testmon-1.2.0
collected 0 items

============================ no tests ran in 0.04s ============================
non-zero exit-code
pytest --testmon
============================= test session starts =============================
platform win32 -- Python 3.8.10, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
testmon: new DB, environment: default
rootdir: C:\Users\kftf964\playground\testmon\test_dir
plugins: testmon-1.2.0
collected 1 item

tests\test_script.py .                                                   [100%]

============================== 1 passed in 0.20s ==============================

Windows users with Git bash installed can use this shell script to reproduce:

# Configure to see all output
set -o verbose
set -e

# Set up project
mkdir source
touch source/__init__.py
printf "def fun():\n    print('a')" > source/script.py
mkdir tests
touch tests/__init__.py
printf "from source.script import fun\n\ndef test_fun():\n    fun()" > tests/test_script.py
pip install -U pytest pytest-testmon==1.2.0

# First run, tests are executed
pytest --testmon

# Change code
printf "def fun():\n    print('b')" > source/script.py

# No tests are executed the first time
pytest --testmon || echo "non-zero exit-code"

# Tests are executed
pytest --testmon

It works fine on a Docker image that I tried, so I suspect the issue is related e.g. to Windows paths using backslashes.

tarpas commented 1 year ago

@robertschweizer https://stackoverflow.com/questions/31519880/windows-compatible-filesystems-file-time-resolutions says that FAT16 and FAT32 filesystems have 2s file modification time resolution. Are you using FAT? Does your bash git script reproduce the error if you insert 2s delay after first pytest run? ( @secult )

tarpas commented 1 year ago

@robertschweizer https://stackoverflow.com/questions/31519880/windows-compatible-filesystems-file-time-resolutions says that FAT16 and FAT32 filesystems have 2s file modification time resolution. Are you using FAT? Does your bash git script reproduce the error if you insert 2s delay after first pytest run? ( @secult )

@robertschweizer @secult ignore that, I realized the problem is something else.