search

thp / urlwatch

Watch (parts of) webpages and get notified when something changes via e-mail, on your phone or via other means. Highly configurable.
https://thp.io/2008/urlwatch/
Other
2.81k stars 352 forks source link

Parametrized test names are not stable across versions #676

Closed sbraz closed 2 years ago

sbraz commented 2 years ago

Hi, I noticed that you append a number to all parametrized tests. However, this number changes when you add a new test. For instance, lib/urlwatch/tests/test_filter_documentation.py::test_url[https://example.net/ocr-test.png-job28] used to be lib/urlwatch/tests/test_filter_documentation.py::test_url[https://example.net/ocr-test.png-job27] (and 26) in older versions.

We at Gentoo maintain a list of skipped test which we often have to update because of this: https://github.com/gentoo/gentoo/blob/24396dbde5e3e7fafd5f3a1ff91be6ea8b88cd8c/www-misc/urlwatch/urlwatch-2.23.ebuild#L39

Could you please look for a way to make the names more stable?

thp commented 2 years ago

This is implemented now, thanks for noticing (this means that with the next release, the ignored tests probably break once more (as the IDs are removed), but then it's stable.

sbraz commented 2 years ago

Very nice, thanks!