search

jerry-git / pytest-split

Pytest plugin which splits the test suite to equally sized "sub suites" based on test execution time.
https://jerry-git.github.io/pytest-split/
MIT License
246 stars 30 forks source link

pytest-split test collection is not compatible with pytest-testmon #93

Open marcodicro-dp opened 6 months ago

marcodicro-dp commented 6 months ago

Is pytest-split compatible with pytest-testmon?

I want ro run all tests in groups (using pytest-split) and then use testmon to run only the affected tests inside those groups. Testmon said no files have changed (no affected tests) but is pytest-split is running all tests anyways:

testmon: changed files: 0, unchanged files: 456, environment: default rootdir: /Users/marcodicro/src/repo configfile: test/project/pytest.ini plugins: flaky-3.8.1, testmon-2.1.1, split-0.8.2 collecting 5771 items

[pytest-split] Splitting tests with algorithm: duration_based_chunks [pytest-split] Running group 1/25 (estimated duration: 231.00s)

collected 5772 items / 5541 deselected / 231 selected

marcodicro-dp commented 3 months ago

This issue is still happening, now happened to me while trying to build testmon db running my tests in sequential groups using pytest-split:

pytest tests --testmon-noselect --splits 25 --group 1 -v --ignore=tests/analytics_es_tests/ --rootdir=/__w/sample_repo/sample_repo/test/app/../..

Groups were wrong, tests were repeated between the groups.