adamws / kicad-kbplacer

KiCad plugin for automatic keyboard's key placement and routing
GNU General Public License v3.0
333 stars 23 forks source link

GUI state tests fail randomly #11

Closed adamws closed 6 months ago

adamws commented 11 months ago

Some of the tests, for example test_gui_state_restore are flaky and fail on circleci randomly, with errors like this:

tmpdir = local('/tmp/pytest-of-root/pytest-0/test_gui_state_restore_no_addi0')
screen_manager = <tests.test_gui.LinuxVirtualScreenManager object at 0x7faae1481410>
window_name = 'kbplacer'
gui_callback = <function test_gui_state_restore.<locals>._callback at 0x7faae147a7a0>

    def run_gui_test(tmpdir, screen_manager, window_name, gui_callback) -> None:
        is_ok = True
        # for some reason, it occasionally may fail with
        # 'wxEntryStart failed, unable to initialize wxWidgets!' error, most likely
        # this is not related with plugin's code - try to get screenshot 3 times
        # to limit false positives
        max_attempts = 3
        for i in range(0, max_attempts):
            with screen_manager as mgr:
                p = gui_callback()
                is_ok = mgr.screenshot(window_name, f"{tmpdir}/screenshot.png")
                try:
                    outs, errs = p.communicate("q\n", timeout=1)
                except subprocess.TimeoutExpired:
                    p.kill()
                    outs, errs = p.communicate()

                assert outs == "Press any key to exit: "
>               assert errs == ""
E               AssertionError: assert 'double free ...ion (!prev)\n' == ''
E                 + double free or corruption (!prev)

tests/test_gui.py:224: AssertionError

This can be reproduced when running tests in docker container (just like circleci does it). Most likely this is the problem with tests because no problems has been observed when running this directly on host.