search

gazebo-tooling / release-tools

8 stars 9 forks source link

Make test to fail if problems where found in CTest results (for #1130) #1132

Closed j-rivero closed 5 months ago

j-rivero commented 5 months ago

While testing #1130 using gz-fuel-tools9 ASAN branch just noticed that although we are injecting missing context into some tests, in the case of finding ASAN failures, the context is indeed injected but the failure state is not be injected so the tests are still passing all of them even if the status was "fail" in CTest.

With this PR I'm using the field "errors" reserved for non-CI strictly failing reasons to inject the errors from CTest into the Gtest testsuite. The PR also adds a bit of output about what we are doing and where.

See it working in https://build.osrfoundation.org/job/_test_gz_fuel_tools-ci_asan-gz-fuel-tools9-jammy-amd64/17/

j-rivero commented 5 months ago

Tested also here, good results for gz-transport asan https://build.osrfoundation.org/view/gz-ionic/job/gz_transport-ci_asan-main-jammy-amd64/32/

azeey commented 5 months ago

Not sure why the test names have <ini> in them in https://build.osrfoundation.org/view/gz-ionic/job/gz_transport-ci_asan-main-jammy-amd64/32/, but this LGTM! image

j-rivero commented 5 months ago

Not sure why the test names have <ini> in them in https://build.osrfoundation.org/view/gz-ionic/job/gz_transport-ci_asan-main-jammy-amd64/32/, but this LGTM!

Looking into the XML attributes I can not see anything obvious that can trigger this. I agree that it is not a blocker