search

stryker-mutator / stryker-net

Mutation testing for .NET core and .NET framework!
https://stryker-mutator.io
Apache License 2.0
1.76k stars 175 forks source link

Azure DevOps extension to integrate html report #394

Closed rouke-broersma closed 4 years ago

rouke-broersma commented 5 years ago

In Azure DevOps it is possible to add custom build report tabs through extensions. If we create a stryker extension we could load the whole html report into azure devops.

rouke-broersma commented 5 years ago

We may want this to be a separate repo, as the html report is not stryker-net specific so any mutation report could be loaded by azure devops

anhaehne commented 5 years ago

As an intermediate solution to integrate with Azure Pipelines we could add a reporter that emits Errors in the form of: ##vso[task.LogIssue type=error;]This is the error

See: Azure Pipelines - Write an error

That would allow fail a build when a mutant survives.

rouke-broersma commented 5 years ago

A pipeline should already fail when Stryker is properly configured as we exit Stryker with a non-zero status code if the amount of survived mutants is over the threshold.

On Wed, Jul 31, 2019, 20:10 Andre Hähnel notifications@github.com wrote:

As an intermediate solution to integrate with Azure Pipelines we could add a reporter that emits Errors in the form of: ##vso[task.LogIssue type=error;]This is the error

See: Azure Pipelines - Write an error https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/powershell?view=azure-devops#write-an-error

That would allow fail a build when a mutant survives.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/stryker-mutator/stryker-net/issues/394?email_source=notifications&email_token=AA3N5HFMP4MGNP5CWUVARY3QCHIQ5A5CNFSM4G4LNNF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3IC5RA#issuecomment-516959940, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3N5HDDKAOHBSJ4RANIGV3QCHIQ5ANCNFSM4G4LNNFQ .

anhaehne commented 5 years ago

Yes but the user will see an error message like dotnet.exe has exited with an non zero exit code and would have to drill down into the logs to see which mutants have survived.

With a special reporter, the pipeline result would display each survived mutant as an error.

rouke-broersma commented 5 years ago

Mhm good point 👍

On Wed, Jul 31, 2019, 20:50 Andre Hähnel notifications@github.com wrote:

Yes but the user will see an error message like dotnet.exe has exited with an non zero exit code and would have to drill down into the logs to see which mutants have survived.

With a special reporter, the pipeline result would display each survived mutant as an error.

Holen Sie sich Outlook für Androidhttps://aka.ms/ghei36

From: Rouke Broersma notifications@github.com Sent: Wednesday, July 31, 2019 8:13:21 PM To: stryker-mutator/stryker-net stryker-net@noreply.github.com Cc: Andre Hähnel andrehaehnel@hotmail.de; Comment < comment@noreply.github.com> Subject: Re: [stryker-mutator/stryker-net] Azure DevOps extension to integrate html report (#394)

A pipeline should already fail when Stryker is properly configured as we exit Stryker with a non-zero status code if the amount of survived mutants is over the threshold.

On Wed, Jul 31, 2019, 20:10 Andre Hähnel notifications@github.com wrote:

As an intermediate solution to integrate with Azure Pipelines we could add a reporter that emits Errors in the form of: ##vso[task.LogIssue type=error;]This is the error

See: Azure Pipelines - Write an error < https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/powershell?view=azure-devops#write-an-error

That would allow fail a build when a mutant survives.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub < https://github.com/stryker-mutator/stryker-net/issues/394?email_source=notifications&email_token=AA3N5HFMP4MGNP5CWUVARY3QCHIQ5A5CNFSM4G4LNNF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3IC5RA#issuecomment-516959940 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AA3N5HDDKAOHBSJ4RANIGV3QCHIQ5ANCNFSM4G4LNNFQ

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub< https://github.com/stryker-mutator/stryker-net/issues/394?email_source=notifications&email_token=ABJOUKQNO7XH3RGPYJEGYODQCHI4DA5CNFSM4G4LNNF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3IDF2I#issuecomment-516961001>, or mute the thread< https://github.com/notifications/unsubscribe-auth/ABJOUKTAXUS35GOFKDDYF5TQCHI4DANCNFSM4G4LNNFQ

.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/stryker-mutator/stryker-net/issues/394?email_source=notifications&email_token=AA3N5HEOG5YZ4K4HZJZL4HTQCHNFZA5CNFSM4G4LNNF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3IGRRQ#issuecomment-516974790, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3N5HA3YHNHCCOTX3JKHW3QCHNFZANCNFSM4G4LNNFQ .

rouke-broersma commented 4 years ago

It's live! https://marketplace.visualstudio.com/items?itemName=stryker-mutator.mutation-report-publisher