Dialog test fail in GitHub Action

I created some end to end test that work correctly in my local machine (Win11 22H). These tests use a fixture that manage the instances of QApplication and the main QMainWidget. The goal is to load the aplication and perform some setup actions. This fixture returns the instance of QApplication. In each test the QMainWidget instance is accesed using QApplication.activeWindow() method.

When tests are executed by GitHub action the QApplication.activeWindow() returns None. I tried returning the QMainWidget instance but it looks like there is an error in its creation.

 =================================== FAILURES ===================================
| ____________________________________ test_1 ____________________________________
|
| qtbot = <pytestqt.qtbot.QtBot object at 0x7f5f6fe22d40>
| app = <PyQt6.QtWidgets.QApplication object at 0x7f5f6fe15e10>
|
| >   ???
| E   AttributeError: 'NoneType' object has no attribute 'button'

I reproduced the issue creating an script that follows the application structure

Simple app with dialog:

import sys
from PyQt6.QtWidgets import QApplication, QWidget, QLabel, QDialog, QDialogButtonBox, QPushButton

class Window(QWidget):
    def __init__(self):
        super().__init__()
        self.setWindowTitle("Hello World")
        self.setGeometry(100, 100, 280, 80)
        self.label = QLabel("Hello World", self)
        self.button = QPushButton("Open dialog", self)
        self.button.clicked.connect(lambda: Dialog.show_dialog())
        self.current_dialog = None

class Dialog(QDialog):
    def __init__(self):
        super().__init__()
        self.setWindowTitle("Hello World dialog")
        self.setGeometry(100, 100, 280, 80)
        self.button_box = QDialogButtonBox(QDialogButtonBox.StandardButton.Ok, self)
        self.button_box.accepted.connect(self.accept_dialog)

    @staticmethod
    def show_dialog():
        app: QApplication = QApplication.instance()
        active_window = app.activeWindow()
        dialog = Dialog()
        active_window.current_dialog = dialog
        dialog.exec()

    def accept_dialog(self):
        self.close()
        self.deleteLater()

if __name__ == '__main__':
    app = QApplication(sys.argv)
    win = Window()
    win.show()
    sys.exit(app.exec())

Test:

import pytest
from PyQt6.QtCore import Qt, QTimer
from PyQt6.QtWidgets import QApplication, QDialogButtonBox
from pyqt.__main__ import Window

@pytest.fixture(scope="function")
def app(qtbot):
    test_app: QApplication = QApplication.instance()
    if test_app is None:
        test_app = QApplication([])

    window: Window = test_app.activeWindow()
    if window is not None:
        window.close()

    # Application reset

    window = Window()
    window.show()

    with qtbot.waitExposed(window):
        yield test_app

    test_app.quit()

def test_1(qtbot, app):
    window: Window = app.activeWindow()
    dialog_handled = False

    def handle_dialog():
        dialog = window.current_dialog
        while not dialog:
            dialog = window.current_dialog
        dialog.button_box.button(QDialogButtonBox.StandardButton.Ok).click()

        nonlocal dialog_handled
        dialog_handled = True

    QTimer.singleShot(5, handle_dialog)
    qtbot.mouseClick(window.button, Qt.MouseButton.LeftButton)

    assert dialog_handled

For testing purposes I am using act in order to test locally the actions. To completly reproduce the failure using act some extra dependencies must be installed (xvfb libnss3 libxdamage1 libasound2). Workflow:

name: Pytest report

on:
  push:  
    branches:  
      - main  
      - develop
  pull_request:  
    branches:  
      - main

jobs:  
  build:  
    runs-on: ubuntu-latest  
    strategy:  
      matrix:
        python-version: ["3.10"]
    env:
      DISPLAY: ':99.0'

    steps:  
      - uses: actions/checkout@v3  
      - name: Set up Python ${{ matrix.python-version }}  
        uses: actions/setup-python@v4  
        with:  
          python-version: ${{ matrix.python-version }}  

      - uses: tlambert03/setup-qt-libs@v1
      - name: Ubuntu setup for pytest-qt
        run: |  
          sudo apt-get install -y xvfb libnss3 libxdamage1 libasound2
          /sbin/start-stop-daemon --start --quiet --pidfile /tmp/custom_xvfb_99.pid --make-pidfile --background --exec /usr/bin/Xvfb -- :99 -screen 0 1920x1200x24 -ac +extension GLX

      - name: Python dependencies installation  
        run: |  
          python -m pip install --upgrade pip  
          pip install -r requirements.txt

      - name: Test with pytest and generate report
        run: |  
          pytest --cov-report xml --cov

      - name: Upload Coverage Report
        uses: codacy/codacy-coverage-reporter-action@v1
        with:
          project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}
          coverage-reports: coverage.xml

Here is the original workflow error from GitHub in the original repository.

I managed to partially solve the error including the fixture code inside the test. It only worked for one of the tests in the original repository (test_open_project). The other tests, as well as the example reproduction, completly freeze the console (both local and remote) when this "workaround" is applied. test_open_project test is the only one that does not deal with a custom QDialog but a QFileDialog with mocked features.

I am new to the qt world so I am bit confused with the life cycle management of the widgets and dialog management when testing. The workaround for dialog testing was taken from #237 .

Here it is the list of python libraries used in the project:

attrs==22.1.0
colorama==0.4.5
iniconfig==1.1.1
lxml==4.9.1
lxml-stubs==0.4.0
packaging==21.3
pluggy==1.0.0
py==1.11.0
pyparsing==3.0.9
pytest==7.1.3
pytest-mock==3.11.1
pytest-lazy-fixture==0.6.3
pytest-qt==4.2.0
pytest-xvfb==3.0.0
coverage==7.2.7
pytest-cov==4.1.0
shortuuid==1.0.9
StrEnum==0.4.8
tomli==2.0.1
decorator==5.1.1
mypy==0.982
mypy-extensions==0.4.3
typing_extensions==4.4.0
validators==0.20.0
PyQt6==6.5.0
PyQt6-Qt6==6.5.0
PyQt6-sip==13.5.1
PyQt6-WebEngine==6.5.0
PyQt6-WebEngine-Qt6==6.5.1
pyyaml==6.0
markdown==3.4.3

pytest-dev / pytest-qt

Dialog test fail in GitHub Action #512