Failure during Act run is not "raised" in test output

[JoshMcCullough]

Describe the bug If an error occurs (from Act) while running a test, no info is shown without capturing/showing the log.

To Reproduce

1. Introduce a failure within an action's YML which is under test. (For instance, add a step with a uses which points to a non-existent file path e.g. step.should-fail.uses: ./path/to/no/dir
2. Run tests e.g. npm test.

The test may show as passed when actually Act failed under the hood. See:

Expected behavior If Act fails during a test run, the test should fail (e.g. an error should be thrown).

Logs See above.

[JoshMcCullough]

Top be clear, this is "aviodable" if we always run the tests with logging enabled, but if this was run by CI then it would pass when it should have failed.

[shubhbapna]

I am not sure if I would like the test to fail if the workflow fails because users might want to test for failures in their workflow as well.

I think the way to cause the test to fail would be to test for success for that step. So something like this should be added to the test file:

const result = await act.runEvent("push")
expect(result[0].status).toBe(0) // 0 indicates success. so if the step failed then this will cause the test to fail as well

[shubhbapna]

Closing this, feel free to open it again if anyone prefers some other way to do this