microsoft / codecoverage

MIT License
84 stars 11 forks source link

dotnet-coverage connect returns successful exit code even if subprocess fails #133

Closed mvromer closed 2 months ago

mvromer commented 2 months ago

This seems to be similar to an issue raised two years ago in regard to dotnet-coverage collect exhibiting the same behavior. That issue was raised and addressed here.

In my situation, I have an Azure Pipeline where I'm starting a coverage collection server with dotnet-coverage collect --server-mode --background --session-id {session} in one step. Later in another step I will run dotnet-coverage connect {session} dotnet test to kick off all my unit and integration tests and gather coverage data.

What I've noticed is that this latter command will always return with an exit status of 0, even if dotnet test returns non-zero when at least one of the tests fail. This means I can't stop and fail the pipeline run on test failure like I used to prior to incorporating dotnet-coverage in my pipelines. I've been able to confirm this behavior locally by doing a similar setup except instead of calling dotnet test I instead will call a .NET console app that simply returns an exit status of 1. In that case, the exit status of dotnet-coverage connect is still zero.

I think if dotnet-coverage connect runs the client not in background mode, then it should relay the exit status of the client subprocess. This would allow for automated tests and pipelines to properly detect if the underlying client/test runner fails and to respond appropriately.

fhnaseer commented 2 months ago

This has been fixed and will be available in the next release of dotnet-coverage (17.12.4).