tmux-python / tmuxp

🖥️ Session manager for tmux, build on libtmux.
https://tmuxp.git-pull.com/
MIT License
4.04k stars 231 forks source link

1.34.0: test_environment_variables fails on really fast CI runners #902

Open sdelafond opened 7 months ago

sdelafond commented 7 months ago

Step 1: Provide a summary of your problem

I'm the Debian maintainer for tmuxp, and on fast amd64 and s390x CI hosts, test_environment_variables sometimes fails. This happens in different locations, but always on a line that asserts pane.capture_pane.

This is Debian bug https://bugs.debian.org/1060775

Step 2: Provide tmuxp details

-------------------------
environment:
        dist: Linux-6.6.8-amd64-x86_64-with-glibc2.37
        arch: x86_64
        uname: Linux; hulk; 6.6.8-amd64
        version: #1 SMP PREEMPT_DYNAMIC Debian 6.6.8-1 (2023-12-22)
-------------------------
python version: 3.11.7 (main, Dec  8 2023, 14:22:46) [GCC 13.2.0]
system PATH: /usr/local/bin:/usr/local/sbin:/bin:/usr/bin:/sbin:/usr/sbin
tmux version: 3.3
libtmux version: 0.25.0
tmuxp version: 1.34.0
tmux path: /bin/tmux
tmuxp path: /usr/lib/python3/dist-packages/tmuxp

Step 3: Describe the problem:

I've tried raising the timeout in that test from 0.3s to 3s but we still see failures:

https://ci.debian.net/packages/t/tmuxp/testing/s390x/41861855/

Steps to reproduce:

I can't reproduce this on my personal development machine, unfortunately.

Observed Results:

Here's the failure log:

=== python3.11 ===
============================= test session starts ==============================
platform linux -- Python 3.11.7, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/autopkgtest-lxc.1o9ufl7q/downtmp/autopkgtest_tmp
plugins: libtmux-0.25.0, rerunfailures-12.0, hypothesis-6.92.2, mock-3.12.0
collected 194 items / 2 deselected / 192 selected

tests/test_plugin.py ..........                                          [  5%]
tests/test_shell.py ..                                                   [  6%]
tests/test_util.py .......                                               [  9%]
tests/cli/test_cli.py ........                                           [ 14%]
tests/cli/test_convert.py ........                                       [ 18%]
tests/cli/test_debug_info.py .                                           [ 18%]
tests/cli/test_freeze.py ......                                          [ 21%]
tests/cli/test_import.py .......                                         [ 25%]
tests/cli/test_load.py ...............s........                          [ 38%]
tests/cli/test_ls.py .                                                   [ 38%]
tests/cli/test_shell.py ....................                             [ 48%]
tests/tests/test_helpers.py ..                                           [ 50%]
tests/workspace/test_builder.py ...s......Fs............................ [ 70%]
..........                                                               [ 76%]
tests/workspace/test_config.py ..............                            [ 83%]
tests/workspace/test_finder.py ..................                        [ 92%]
tests/workspace/test_freezer.py ...                                      [ 94%]
tests/workspace/test_import_teamocil.py ........                         [ 98%]
tests/workspace/test_import_tmuxinator.py ...                            [100%]

=================================== FAILURES ===================================
__________________________ test_environment_variables __________________________

session = Session($1 libtmux_m6iuy9kl)

    @pytest.mark.skipif(
        has_lt_version("3.0"),
        reason="needs -e flag for new-window and split-window introduced in tmux 3.0",
    )
    def test_environment_variables(
        session: Session,
    ) -> None:
        """Test setting of environmental variables in tmux via workspace builder."""
        workspace = ConfigReader._from_file(
            test_utils.get_workspace_file("workspace/builder/environment_vars.yaml")
        )
        workspace = loader.expand(workspace)

        builder = WorkspaceBuilder(session_config=workspace, server=session.server)
        builder.build(session)
        # Give slow shells some time to settle as otherwise tests might fail.
        # Debian: raise timeout from 0.3s to 3s to workaround flakyness on really
        # fast Debian CI workers (see https://bugs.debian.org/1060775)
        time.sleep(3.0)

        assert session.getenv("FOO") == "SESSION"
        assert session.getenv("PATH") == "/tmp"

        no_overrides_win = session.windows[0]
        pane = no_overrides_win.panes[0]
        pane.send_keys("echo $FOO")
>       assert pane.capture_pane()[1] == "SESSION"
E       IndexError: list index out of range

tests/workspace/test_builder.py:410: IndexError

Here's a couple of other failed runs, prior to changing the timeout (some of them happened with tmuxp 1.31, as the problem has been present for a while):

https://ci.debian.net/packages/t/tmuxp/testing/amd64/40953851/ https://ci.debian.net/packages/t/tmuxp/testing/amd64/40904252/ https://ci.debian.net/packages/t/tmuxp/testing/s390x/41855336/ https://ci.debian.net/packages/t/tmuxp/testing/s390x/41828952/ https://ci.debian.net/packages/t/tmuxp/testing/s390x/41730238/

Expected Results:

test_environment_variables reliably passes.