ros2 / rosidl

Packages which provide the ROS IDL (.msg) definition and code generation.
Apache License 2.0
75 stars 125 forks source link

:farmer: `test_multi_nested_message_introspection` flaky test in windows repeated #782

Open Crola1702 opened 8 months ago

Crola1702 commented 8 months ago

Bug report

Required Info:

Steps to reproduce issue

  1. Run a build in nightly_win_rep
  2. See tests fail

Additional information

Reference build: https://ci.ros2.org/view/nightly/job/nightly_win_rep/3235/

Test regressions:

Log output: ``` C:\ci\ws\src\ros2\rosidl\rosidl_typesupport_introspection_tests\test\test_multi_nested_message_introspection.cpp:214 Expected equality of these values: item_reference Which is: 736-byte object <50-01 0E-15 A1-01 00-00 C0-FB E1-AB A1-01 00-00 0B-00 00-00 00-00 00-00 FC-A8 F7-E1 00-00 00-00 3D-00 00-00 00-00 00-00 D0-8B 11-15 A1-01 00-00 00-00 00-00 00-00 00-E0 00-00 00-00 00-00 00-00 ... 01-00 00-00 00-00 00-00 20-EA 11-15 A1-01 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 20-E6 11-15 A1-01 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 65-00 00-00 FF-7F 00-00> message.array_of_arrays[i] Which is: 736-byte object <50-01 0E-15 A1-01 00-00 C0-78 11-15 A1-01 00-00 0B-00 00-00 00-00 00-00 FC-A8 F7-E1 00-00 00-00 3D-00 00-00 00-00 00-00 D0-8B 11-15 A1-01 00-00 00-00 00-00 00-00 00-E0 00-00 00-00 00-00 00-00 ... 01-00 00-00 00-00 00-00 20-54 10-15 A1-01 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 A0-54 10-15 A1-01 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 65-00 00-00 00-00 00-00> C:\ci\ws\src\ros2\rosidl\rosidl_typesupport_introspection_tests\test\test_multi_nested_message_introspection.cpp:214 Expected equality of these values: item_reference Which is: 736-byte object <00-00 00-00 00-00 00-00 0A-FB E1-AB 00-2C 00-80 6F-12 83-C0 CA-21 09-40 00-00 00-00 00-00 00-00 6F-12 83-C0 CA-21 09-C0 04-3A 31-E4 00-2D 00-80 00-00 00-00 FF-FF FF-7F 00-00 00-80 00-00 00-00 ... 01-00 00-00 00-00 00-00 C0-E8 11-15 A1-01 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 A0-E6 11-15 A1-01 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 00-00 00-00 FF-7F 00-00> message.array_of_arrays[i] Which is: 736-byte object <00-00 00-00 00-00 00-00 0A-3A 33-E4 00-2C 00-80 6F-12 83-C0 CA-21 09-40 00-00 00-00 00-00 00-00 6F-12 83-C0 CA-21 09-C0 04-3A 31-E4 00-2D 00-80 00-00 00-00 FF-FF FF-7F 00-00 00-80 00-00 00-00 ... 01-00 00-00 00-00 00-00 60-4E 10-15 A1-01 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 60-52 10-15 A1-01 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00> C:\ci\ws\src\ros2\rosidl\rosidl_typesupport_introspection_tests\test\test_multi_nested_message_introspection.cpp:256 Expected equality of these values: message Which is: 6864-byte object <50-01 0E-15 A1-01 00-00 C0-78 11-15 A1-01 00-00 0B-00 00-00 00-00 00-00 FC-A8 F7-E1 00-00 00-00 3D-00 00-00 00-00 00-00 D0-8B 11-15 A1-01 00-00 00-00 00-00 00-00 00-E0 00-00 00-00 00-00 00-00 ... 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 80-BE 11-15 A1-01 00-00 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 80-C1 11-15 A1-01 00-00 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00> message_copy Which is: 6864-byte object <50-01 0E-15 A1-01 00-00 C0-FB E1-AB A1-01 00-00 0B-00 00-00 00-00 00-00 FC-A8 F7-E1 00-00 00-00 3D-00 00-00 00-00 00-00 D0-8B 11-15 A1-01 00-00 00-00 00-00 00-00 00-E0 00-00 00-00 00-00 00-00 ... 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 20-20 12-15 A1-01 00-00 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 50-36 12-15 A1-01 00-00 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00> ```

Flaky ratio:

job_name last_fail first_fail build_count failure_count failure_percentage
nightly_win_rep 2024-01-16 2023-12-19 28 7 25.0
nightly_linux-aarch64_repeated 2024-01-04 2024-01-04 31 1 3.23

aarch64 repeated reference build

First time it was seen was on 2022-02-03, so it's been failing with a 25-30% rate since then.