seleniumbase / SeleniumBase

📊 Python's all-in-one framework for web crawling, scraping, testing, and reporting. Supports pytest. UC Mode provides stealth. Includes many tools.
https://seleniumbase.io
MIT License
4.5k stars 916 forks source link

"--list-fail-page" shows duplicate entries sometimes #1671

Closed mdmintz closed 1 year ago

mdmintz commented 1 year ago

--list-fail-page shows duplicate entries sometimes

pytest test_suite.py --list-fail-page --rs
============================================ test session starts ============================================
platform darwin -- Python 3.11.1, pytest-7.2.0, pluggy-1.0.0
rootdir: /Users/michael/github/SeleniumBase/examples, configfile: pytest.ini
plugins: html-2.0.1, forked-1.4.0, rerunfailures-11.0, xdist-3.1.0, ordering-0.6, cov-4.0.0, metadata-2.0.4, seleniumbase-4.11.3
collected 4 items                                                                                           

test_suite.py .
(This test should fail)
F
=> Fail Page: https://xkcd.com/1675/
.
=> Fail Page: https://xkcd.com/1675/

(This test should fail)
F
=> Fail Page: https://xkcd.com/1670/

There, it was shown three times, when it should have only been shown twice. (For 2 expected failures out of 4 tests)

The reason is because sb_config._fail_page wasn't getting reset between tests.

Also note that -v, (verbose mode) will also trigger the --list-fail-page option, (if not running multithreaded tests).


Another issue with --list-fail-page is that sometimes the output doesn't appear until after all tests have completed. (A fail page URL should be displayed after each failing test.)

mdmintz commented 1 year ago

Fixed in 4.11.4 - https://github.com/seleniumbase/SeleniumBase/releases/tag/v4.11.4