Build farmer handoff 2019-03-01

sloretz commented 5 years ago

Outgoing @sloretz from #160 Incoming @dirk-thomas

Known flaky tests and recurring failures

test_params_yaml fails on all platforms #166, #155
[x] linux_nightly_coverage qt_gui_cpp test fails #158
[x] rviz tests are failing on aarch64 linux repeated (original contributors aware) #144
pendulum tests fail on aarch64 opensplice #133
[x] launch testing failures on linux repeated #126
Services/parameter tests with Fast-RTPS are flaky on aarch64 #136

[x] Lots of test failures on windows debug

[x] Recent increase documented in #167

Failures in https://ci.ros2.org/job/nightly_win_deb/1175

message_filters.test.directed
message_filters.test.test_approxsync
message_filters.test.test_message_filters_cache
projectroot.directed.py
projectroot.gtest_avoid_ros_namespace_conventions_qos__rmw_connext_cpp
projectroot.gtest_avoid_ros_namespace_conventions_qos__rmw_fastrtps_cpp
projectroot.gtest_executor__rmw_connext_cpp
projectroot.gtest_executor__rmw_fastrtps_cpp
projectroot.gtest_local_parameters__rmw_connext_cpp
projectroot.gtest_local_parameters__rmw_fastrtps_cpp
projectroot.gtest_multiple_service_calls__rmw_connext_cpp
projectroot.gtest_multiple_service_calls__rmw_fastrtps_cpp
projectroot.gtest_multithreaded__rmw_connext_cpp
projectroot.gtest_multithreaded__rmw_fastrtps_cpp
projectroot.gtest_publisher__rmw_connext_cpp
projectroot.gtest_publisher__rmw_fastrtps_cpp
projectroot.gtest_services_in_constructor__rmw_connext_cpp
projectroot.gtest_services_in_constructor__rmw_fastrtps_cpp
projectroot.gtest_spin__rmw_connext_cpp
projectroot.gtest_spin__rmw_fastrtps_cpp
projectroot.gtest_subscription__rmw_connext_cpp
projectroot.gtest_subscription__rmw_fastrtps_cpp
projectroot.gtest_timer__rmw_connext_cpp
projectroot.gtest_timer__rmw_fastrtps_cpp
projectroot.gtest_waitable__rmw_connext_cpp
projectroot.gtest_waitable__rmw_fastrtps_cpp
projectroot.message_filters_msg_cache_unittest
projectroot.message_filters_test_approximate_time_policy
projectroot.message_filters_test_chain
projectroot.message_filters_test_exact_time_policy
projectroot.message_filters_test_simple
projectroot.message_filters_test_subscriber
projectroot.message_filters_test_time_sequencer
projectroot.message_filters_time_synchronizer_unittest
projectroot.rqt_py_common
projectroot.test_action_client_server__Fibonacci__rclcpp__rmw_connext_cpp
projectroot.test_action_client_server__Fibonacci__rclcpp__rmw_fastrtps_cpp
projectroot.test_action_client_server__NestedMessage__rclcpp__rmw_connext_cpp
projectroot.test_action_client_server__NestedMessage__rclcpp__rmw_fastrtps_cpp
projectroot.test_approxsync.py
projectroot.test_callback_exceptions
projectroot.test_client
projectroot.test_client_scope_consistency_cpp__rmw_connext_cpp
projectroot.test_client_scope_consistency_cpp__rmw_fastrtps_cpp
projectroot.test_client_scope_cpp__rmw_connext_cpp
projectroot.test_client_scope_cpp__rmw_fastrtps_cpp
projectroot.test_executor
projectroot.test_externally_defined_services
projectroot.test_find_weak_nodes
projectroot.test_lifecycle_node
projectroot.test_message_filters_cache.py
projectroot.test_multi_threaded_executor
projectroot.test_node
projectroot.test_node_global_args
projectroot.test_parameter_server_cpp__rmw_connext_cpp
projectroot.test_parameter_server_cpp__rmw_fastrtps_cpp
projectroot.test_publisher
projectroot.test_publisher_subscription_count_api
projectroot.test_register_custom_callbacks
projectroot.test_server
projectroot.test_service
projectroot.test_services_cpp__rmw_connext_cpp
projectroot.test_services_cpp__rmw_fastrtps_cpp
projectroot.test_state_machine_info
projectroot.test_subscription
projectroot.test_subscription_publisher_count_api
projectroot.tf2_ros_test_message_filter
rclcpp.cppcheck.warning: nullPointer (src\rclcpp\clock.cpp:139)
rmw_connext_shared_cpp.cppcheck.error: unknownMacro (C:\J\workspace\nightly_win_deb\ws\src\ros2\rmw_connext\rmw_connext_shared_cpp\include\rmw_connext_shared_cpp\wait.hpp:51)
rmw_connext_shared_cpp.cppcheck.error: unknownMacro (include\rmw_connext_shared_cpp\wait.hpp:51)
rqt_py_common.test.test_rqt_ros_graph
sros2.test.test_policy_to_permissions
test_communication.build.test_communication.test_action_client_server__Fibonacci__rclcpp__rmw_connext_cpp_Debug.test_action_client_server
test_communication.build.test_communication.test_action_client_server__Fibonacci__rclcpp__rmw_fastrtps_cpp_Debug.test_action_client_server
test_communication.build.test_communication.test_action_client_server__NestedMessage__rclcpp__rmw_connext_cpp_Debug.test_action_client_server
test_communication.build.test_communication.test_action_client_server__NestedMessage__rclcpp__rmw_fastrtps_cpp_Debug.test_action_client_server
test_rclcpp.build.test_rclcpp.test_client_scope_consistency_cpp__rmw_connext_cpp_Debug.test_client_scope_consistency_cpp
test_rclcpp.build.test_rclcpp.test_client_scope_consistency_cpp__rmw_fastrtps_cpp_Debug.test_client_scope_consistency_cpp
test_rclcpp.build.test_rclcpp.test_client_scope_cpp__rmw_connext_cpp_Debug.test_client_scope_cpp
test_rclcpp.build.test_rclcpp.test_client_scope_cpp__rmw_fastrtps_cpp_Debug.test_client_scope_cpp
test_rclcpp.build.test_rclcpp.test_parameter_server_cpp__rmw_connext_cpp_Debug.test_parameter_server_cpp
test_rclcpp.build.test_rclcpp.test_parameter_server_cpp__rmw_fastrtps_cpp_Debug.test_parameter_server_cpp
test_rclcpp.build.test_rclcpp.test_services_cpp__rmw_connext_cpp_Debug.test_services_cpp
test_rclcpp.build.test_rclcpp.test_services_cpp__rmw_fastrtps_cpp_Debug.test_services_cpp
tf2_ros.tf2_ros_test_message_filter.gtest.missing_result

Many instances of stuck colcon/build processes after a previous job fails on windows #161
- killing the process tree fixes the issue

Outstanding buildfarmer tasks

Periodic upgrades on Mac and Windows
- mini1/mini2/mini3/icecube/portable upgraded Feb 28, 2019
- windshield upgraded Mar 01, 2019
[x] Profile memory usage of bridge compilation job #106 (failures currently resolved by increasing instance size)

dirk-thomas commented 5 years ago

Installed OpenSplice 6.9.190227OSS (r2, updated tag/release from today) on all macOS and Windows nodes.

dirk-thomas commented 5 years ago

Installed OpenSplice 6.9.190403OSS on all macOS and Windows nodes.

dirk-thomas commented 5 years ago

ros2/rclcpp#682 fixes 12 test failures on Windows debug.

dirk-thomas commented 5 years ago

Installed Visual Studio 2019 on all Windows nodes.

dirk-thomas commented 5 years ago

Upgraded CMake to version 3.14.1 on all Windows nodes.

dirk-thomas commented 5 years ago

Updated TightVNC server to 2.8.11 on icecube :crossed_fingers:

dirk-thomas commented 5 years ago

Upgraded OpenCV to 3.4.6 (for VS 2017 and VS 2019) on all Windows nodes.

dirk-thomas commented 5 years ago

Closing in favor of #178.

ros2 / build_farmer

Build farmer handoff 2019-03-01 #168

Known flaky tests and recurring failures

Outstanding buildfarmer tasks