search

microsoft / CSEDevOps

Azure DevOps extensions from CSE DevOps team
MIT License
26 stars 11 forks source link

DevOps Publish Test Results Task only publishes ZAP failures #15

Open devsbryan opened 3 years ago

devsbryan commented 3 years ago

Hi,

The plugin is great, really useful to automate the use of the ZAP scan. The only issue I'm having is with the reporting.

I've eventually got the reporting to NUnit format working by using the bash scripts you have provided.

However the test results published don't seem to match up with the ZAP Scan.

The log of the ZAP scan task showing 114 passing tests, and one warning on a full scan.

But the test results are only showing 2 failed tests and 0 passed (which gives a pass rate of 0% which is obviously not right).

Could you help me with this? Is it a case of changing the handlebar formatting somehow?

image

image

gudbrand3 commented 3 years ago

I have the same behaviour only getting the failed tests which gives a unclear picture of the actual number of tests run. Anyone else figured out how to include the PASSed ones?

Love the plugin btw :) Thanks!

mgrandillo commented 2 years ago

Hi guys! I am having the same issue. Informational tests are reported as errors in ADO pipeline (tests tab). However when running the tests with the zap standalone tool, the messages are reported as warnings. Also, in ADO pipeline (tests tab), we do not see any passed tests, which the logs show that there are many. Is this a bug?