irmen / Pyro5

Pyro 5 - Python remote objects
https://pyro5.readthedocs.io
MIT License
303 stars 36 forks source link

fix flaky test on test_pyro4compat.py::test_compat_config #57

Closed lonly7star closed 2 years ago

lonly7star commented 2 years ago

This PR aims to fix the flaky test on test_pyro4compat.py::test_compat_config so the test could pass single run, multiple test run, and random test run.

The result

The test can pass when running only once, but fail when running the test suit multiple times. Starting on the second run, the following error rased

>       assert conf["NS_PORT"] == 9090
E       assert 12345 == 9090
tests/test_pyro4compat.py:9: AssertionError

Steps to reproduce the issue

  1. install pytest flaky finder with pip install pytest-flakefinder
  2. run pytest with flake-finder with command pytest -k tests/test_pyro4compat.py --flake-finder

Issue of the code

The test tried to check the configuration of Pyro4. Pyro4.config.NS_PORT. It assumes the initial value of it be 9090 and change the value to 12345 later to assert it again but didn't reset the test, which is why it fails start on the second runs

Proposed solution

Adding one line at the end of the function that reset the value to 9090 yro4.config.NS_PORT = 9090 and the test now can pass with arbitrary numbers of re-runs.

irmen commented 2 years ago

thanks!