search

FluidityProject / fluidity

Fluidity
http://fluidity-project.org
Other
366 stars 115 forks source link

Jenkins unit test step not producing fail report #285

Closed gnikit closed 3 years ago

gnikit commented 3 years ago

Correct me if I am wrong, but from what I can tell if a unit test fails on Jenkins the unittestharness will not report a failure since it does not produce a .xml report for junit and moreover the Jenkins step will show as passing, which can be dangerous.

stephankramer commented 3 years ago

By my reading, you're right - I thought I'd had failing unittests picked up by jenkins recently, but I'm guessing these were unittest build failures.

The easy fix would be to have unittestharness exit with a nonzero exit value (just like it does optionally in testharness). The nicer fix would be to have it create that junit report of course...

gnikit commented 3 years ago

Okay, @stephankramer I threw together a fix for the junit report generation, I tried to make it as similar as possible to the structure of the regression tests report, but <time> and <stdout> in the final report are a bit different due to the unit test setup being itself different.

Another thing I should note, is that currently the unittestharness when creating the junit report is setup to register warnings as failures. Not sure if you want to keep it that way.

If you want a more granular junit report I could have a closer look at junit_xml and see what I can do.

gnikit commented 3 years ago

Closed via #287