gazebo-tooling / release-tools

8 stars 9 forks source link

Python test failures in ign-math don't show up in jenkins test results #695

Open scpeters opened 2 years ago

scpeters commented 2 years ago

Currently the python tests in ign-math are not generating JUNIT result files, so Jenkins is unable to detect failures of these tests. For example, in https://github.com/ignitionrobotics/ign-math/pull/407#issuecomment-1093652832 it was noted that a python test fails on 20.04, and while the GitHub action for Ubuntu Focal shows a failure, the Jenkins job indicates success:

Inspection of the build log shows that the python test has indeed failed, but none of the python tests, whether they pass or fail, are registered in the test_results page.

The following tests FAILED:
    121 - PID_TEST.py (Failed)
scpeters commented 2 years ago

The following stack overflow link indicates that pytest can be used with the --junitxml flag to generate JUNIT result files.

The ament_has_pytest.cmake and ament_add_pytest_test.cmake files may be a useful reference. Perhaps that functionality should be added to ign-cmake?