equalsraf / neovim-qt

Neovim client library and GUI, in Qt5.
https://github.com/equalsraf/neovim-qt/wiki
ISC License
1.85k stars 171 forks source link

GitHub Actions No Windows Test Output #884

Open jgehrig opened 3 years ago

jgehrig commented 3 years ago

The Windows test runs do not display any test output, even when -VV or --output-on-failure arguments are used.

When ctest is run manually inside of the runner via, output is displayed. https://github.com/nelsonjchen/reverse-rdp-windows-github-actions

When tst_* and test_* executables are uploaded as artifacts and run locally, test output is displayed.

Bug in GitHub Actions?

jgehrig commented 3 years ago

Here is an example of this issue...

Actions + Windows lacks ctest failure output: https://github.com/equalsraf/neovim-qt/pull/885/checks?check_run_id=3094273084

AppVeyor + Windows correctly displays ctest output: https://ci.appveyor.com/project/equalsraf/neovim-qt/builds/40031037/job/ddhiyj6vtqyaclt9

The test binaries built by GitHub actions work correctly. You can download the exe, run it locally and observe the correct test output.

When ctest is run manually on the runner (via RDP), you can observe the correct output.

It looks like the GitHub Actions runner is not correctly piping output.

equalsraf commented 2 years ago

I dont think it is a bug, I suspect they are not running at all. Here are two example from a recent PR

  1. in linux it looks ok https://github.com/equalsraf/neovim-qt/runs/5035009356?check_suite_focus=true
  2. in windows it is empty and the execution time is 0 - https://github.com/equalsraf/neovim-qt/runs/5035011280?check_suite_focus=true

Maybe a consequence of this check

      - name: Test
        if: ${{ !startsWith(matrix.runner, 'windows') }}

Starting a PR in a second

jgehrig commented 2 years ago

Maybe a consequence of this check

This was added intentionally to prevent this issue from causing build failures on GitHub Actions.

Although it looks like you discovered this in your PR already. Comment added for reference.