search

ros2 / ros2_tracing

Tracing tools for ROS 2.
https://docs.ros.org/en/rolling/
Apache License 2.0
138 stars 42 forks source link

:farmer: test_pub_sub, test_publisher and test_subscription failures #131

Closed Crola1702 closed 2 weeks ago

Crola1702 commented 3 weeks ago

Description

These tests have been failing for a long time in Iron connext and fastrtps-dynamic.

Reference builds:

Test regressions:

All three contain the same log output:

Log output:

11: =================================== FAILURES ===================================
11: __________________________ TestSubscription.test_all ___________________________
11: /tmp/ws/install_isolated/tracetools_test/lib/python3.10/site-packages/tracetools_test/case.py:110: in setUp
11:     self.assertSetEqual(all_event_names, set(self._event_names))
11: E   AssertionError: Items in the first set but not the second:
11: E   'ros2:rmw_subscription_init'
11: E   'ros2:rmw_take'
11: ----------------------------- Captured stdout call -----------------------------
11: [INFO] [launch]: All log files can be found below /home/buildfarm/.ros/log/2024-08-19-01-58-16-902022-a9e85b8b6b09-444099
11: [INFO] [launch]: Default logging verbosity is set to INFO
11: [INFO] [tracetools_launch.action]: Writing tracing session to: /tmp/session-test-subscription-20240819015816
11: [INFO] [test_ping-1]: process started with pid [444104]
11: [INFO] [test_pong-2]: process started with pid [444106]
11: [test_pong-2] [INFO] [1724061498.067881993] [test_pong]: [output] some random ping string
11: [test_ping-1] [INFO] [1724061498.068240921] [test_ping]: [output] some random pong string
11: [test_pong-2] RTI Connext DDS Non-commercial license is for academic, research, evaluation and personal use only. USE FOR COMMERCIAL PURPOSES IS PROHIBITED. See RTI_LICENSE.TXT for terms. Download free tools at rti.com/ncl. License issued to Non-Commercial User license@rti.com For non-production use only.
11: [test_pong-2] Expires on 00-jan-00 See www.rti.com for more information.
11: [test_pong-2] spinning
11: [INFO] [test_pong-2]: process has finished cleanly [pid 444106]
11: [test_ping-1] RTI Connext DDS Non-commercial license is for academic, research, evaluation and personal use only. USE FOR COMMERCIAL PURPOSES IS PROHIBITED. See RTI_LICENSE.TXT for terms. Download free tools at rti.com/ncl. License issued to Non-Commercial User license@rti.com For non-production use only.
11: [test_ping-1] Expires on 00-jan-00 See www.rti.com for more information.
11: [test_ping-1] spinning
11: [INFO] [test_ping-1]: process has finished cleanly [pid 444104]
11: TRACE DIRECTORY: /tmp/session-test-subscription-20240819015816
11: - generated xml file: /tmp/ws/test_results/test_tracetools/test_subscription.xunit.xml -
11: 
11: ---------- coverage: platform linux, python 3.10.12-final-0 ----------
11: 
11: =========================== short test summary info ============================
11: FAILED test/test_subscription.py::TestSubscription::test_all - AssertionError...
11: ============================== 1 failed in 2.52s ===============================
11: -- run_test.py: return code 1
11: -- run_test.py: verify result file '/tmp/ws/test_results/test_tracetools/test_subscription.xunit.xml'
3/3 Test #11: test_subscription ................***Failed    3.22 sec

The following tests FAILED:
      8 - test_pub_sub (Failed)
      9 - test_publisher (Failed)
     11 - test_subscription (Failed)
christophebedard commented 3 weeks ago

Do we know roughly when these tests started failing?

christophebedard commented 3 weeks ago

Ah I see why they're failing. They probably never passed.

The test_tracetools tests in Iron are simply run against the default rmw implementation. Each test is run once, without RMW_IMPLEMENTATION being set explicitly: https://github.com/ros2/ros2_tracing/blob/b7235059b677eb69f04f640ea305ff4bc5e46119/test_tracetools/CMakeLists.txt#L133-L157. The other rmw implementations are not fully instrumented in Iron and the default rmw implementation for Iron (and Jazzy) is rmw_fastrtps_cpp, so running tests against the other rmw implementations will fail (except maybe for rmw_cyclonedds_cpp).

The simplest fix for Iron here is to skip the tests if the "current" rmw implementation is not rmw_fastrtps_cpp.

christophebedard commented 3 weeks ago

See #132 for Iron.

I'll need to do the same for rmw_fastrtps_dynamic_cpp in Jazzy, see Jci__nightly-fastrtps-dynamic_ubuntu_noble_amd64#43. rmw_connextdds in Jazzy is instrumented.

christophebedard commented 2 weeks ago

This was fixed for Iron in #132 and Jazzy in #134, so I'm closing this.