CI does not fail when command fails

[Malacath-92]

See my tests running here: https://github.com/Malacath-92/shader-toy/actions/runs/117158083

You can see that the tests are failing (i.e. exiting with code 1) but the CI does not fail. I assume this is an issue within xvfb-action not forwarding the return code properly. When downgrading to xvfb-action@v1.0 CI fails as expected https://github.com/Malacath-92/shader-toy/actions/runs/117174757

[asbjornu]

I assume this bug was introduced in 8274116c0e19c96396d0b95186a7fedba17d4059. It was rather disheartening seeing the build green, only to discover it was actually exploding like a nuclear bomb as evident only by inspecting the logs.

[klaygomes]

Why still It was not merged?

[OvidijusParsiunas]

Has this issue been resolved? I have tried this with v1, v1.0, v1.3, v1.4, v1.6 and still do not get the pipeline to fail...

See the "Run Headless Test" section in this link: https://github.com/OvidijusParsiunas/if-inverter/runs/6140450696?check_suite_focus=true

The following is my workflow file: https://github.com/OvidijusParsiunas/if-inverter/blob/main/.github/workflows/push-to-main-branch.yml

[OvidijusParsiunas]

Update to my comment above:

The following code inside the runTest.ts file (provided in the starter extension project) does not get hit when your tests fail, hence process.exit(1) never gets called in the CI.

catch (err) {
    console.error('Failed to run tests');
    process.exit(1);
  }

I fixed this by calling process.exit(1) inside the catch statement of the index.ts file (also provided in the starter project):

 catch (err) {
    console.error(err);
    process.exit(1);
  }

View line in this link.