search

tarpas / pytest-testmon

Selects tests affected by changed files. Executes the right tests first. Continuous test runner when used with pytest-watch.
https://testmon.org
MIT License
800 stars 54 forks source link

Testing django commands with `call_command` causes None to be cached with testmon 1.0.0 #145

Closed TauPan closed 4 years ago

TauPan commented 4 years ago

Hi!

I accidentally upgraded to testmon 1.0.0 yesterday, because I had the impression that it was stable. (Maybe because it's the current version on pypi, see https://pypi.org/project/pytest-testmon/ ... but here on github the latest release is 0.9.18.)

I stumbled across this problem where I'm testing django commands with call_command (as described in https://docs.djangoproject.com/en/2.2/topics/testing/tools/#topics-testing-management-commands ) and this causes None to be stored in the cache.

I've made a small github project to reproduce this. Results are in https://travis-ci.org/TauPan/pytest-testmon-bug-none-in-cache

I didn't add different versions of django, as I'm not sure how to do this on travis (maybe I should use tox) and the bug is reproduced.

TauPan commented 4 years ago

Parametrized django version now, same bug with 2.1 and 2.2 as well.

TauPan commented 4 years ago

The behaviour is not triggered with pytest-testmon 0.9.19 (or lower).

tarpas commented 4 years ago

There was a 1.0.0 alpha and RC and this didn't get reported yet. So 1.0.0 should be stable but this slipped through. Thanks a lot for the thorough report! We'll have a look tomorrow.

I also marked the 1.0.0 release here on Github.

Matlino commented 4 years ago

@TauPan Problem was fixed, new version is on pypi. You can try it out :)

TauPan commented 4 years ago

@TauPan Problem was fixed, new version is on pypi. You can try it out :)

Thanks! I've allowed ~= 1.0.0 in my testrepo and with pytest-testmon 1.0.1 all travis builds go green. \o/

blueyed commented 4 years ago

@TauPan Out of interest: do you use testmon actively on CI, i.e. to run less tests there? If so, can you elaborate on how you're doing this, e.g. with regard to PRs vs master builds etc (do you re-run all tests on master build always for example)?

TauPan commented 4 years ago

@TauPan Out of interest: do you use testmon actively on CI, i.e. to run less tests there?

Nope. I use travis almost exclusively to reproduce complicated github issues.