search

gazebosim / gz-physics

Abstract physics interface designed to support simulation and rapid development of robot applications.
https://gazebosim.org
Apache License 2.0
62 stars 38 forks source link

:farmer: Homebrew main, 7 and 6 dartsim test regressions #662

Closed Crola1702 closed 1 day ago

Crola1702 commented 4 days ago

Environment

Description

Steps to reproduce

  1. Run a build in gz-physics main, 7 or 6 homebrew
  2. See tests failures

Output

gz-physics7 was failing to build because of https://github.com/gazebosim/gz-physics/issues/660. This issue is already fixed. However, we still have multiple test regressions to address

Reference build:

Test regressions: * [ConstructEmptyWorldTestUpToEmptyLink/0.ConstructUpToEmptyWorld](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/ConstructEmptyWorldTestUpToEmptyLink_0/ConstructUpToEmptyWorld/) *[COMMON_TEST_added_mass_dartsim.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/COMMON_TEST_added_mass_dartsim/test_ran/) * [COMMON_TEST_addexternalforcetorque_dartsim.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/COMMON_TEST_addexternalforcetorque_dartsim/test_ran/) * [COMMON_TEST_collisions_dartsim.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/COMMON_TEST_collisions_dartsim/test_ran/) * [COMMON_TEST_detachable_joint_dartsim.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/COMMON_TEST_detachable_joint_dartsim/test_ran/) * [COMMON_TEST_free_joint_features_dartsim.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/COMMON_TEST_free_joint_features_dartsim/test_ran/) * [COMMON_TEST_joint_features_dartsim.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/COMMON_TEST_joint_features_dartsim/test_ran/) * [COMMON_TEST_joint_transmitted_wrench_features_dartsim.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/COMMON_TEST_joint_transmitted_wrench_features_dartsim/test_ran/) * [COMMON_TEST_kinematic_features_dartsim.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/COMMON_TEST_kinematic_features_dartsim/test_ran/) * [COMMON_TEST_link_features_dartsim.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/COMMON_TEST_link_features_dartsim/test_ran/) * [COMMON_TEST_shape_features_dartsim.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/COMMON_TEST_shape_features_dartsim/test_ran/) * [COMMON_TEST_simulation_features_dartsim.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/COMMON_TEST_simulation_features_dartsim/test_ran/) * [COMMON_TEST_world_features_dartsim.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/COMMON_TEST_world_features_dartsim/test_ran/) * [INTEGRATION_DoublePendulum.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/INTEGRATION_DoublePendulum/test_ran/) * [AddedMassFeatures.AddedMass](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/AddedMassFeatures/AddedMass/) * [UNIT_Base_TEST.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/UNIT_Base_TEST/test_ran/) * [UNIT_EntityManagement_TEST.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/UNIT_EntityManagement_TEST/test_ran/) * [UNIT_SDFFeatures_TEST.test_ran](https://build.osrfoundation.org/job/gz_physics-ci-main-homebrew-amd64/59/testReport/junit/(root)/UNIT_SDFFeatures_TEST/test_ran/)

Test failures in log output:

The following tests FAILED:
     19 - COMMON_TEST_added_mass_dartsim (Bus error)
     27 - COMMON_TEST_addexternalforcetorque_dartsim (Bus error)
     43 - COMMON_TEST_collisions_dartsim (Bus error)
     51 - COMMON_TEST_construct_empty_world_dartsim (Failed)
     59 - COMMON_TEST_detachable_joint_dartsim (Bus error)
     67 - COMMON_TEST_free_joint_features_dartsim (Bus error)
     75 - COMMON_TEST_joint_features_dartsim (Bus error)
     91 - COMMON_TEST_joint_transmitted_wrench_features_dartsim (Bus error)
     99 - COMMON_TEST_kinematic_features_dartsim (Bus error)
    107 - COMMON_TEST_link_features_dartsim (SEGFAULT)
    115 - COMMON_TEST_shape_features_dartsim (Bus error)
    123 - COMMON_TEST_simulation_features_dartsim (Bus error)
    131 - COMMON_TEST_world_features_dartsim (Bus error)
    137 - INTEGRATION_DoublePendulum (Bus error)
    168 - UNIT_AddedMassFeatures_TEST (Failed)
    170 - UNIT_Base_TEST (Bus error)
    172 - UNIT_EntityManagement_TEST (SEGFAULT)
    174 - UNIT_SDFFeatures_TEST (Bus error)

See reference builds for complete test output. This is probably happening after #660 fix

scpeters commented 4 days ago

I believe these regressions are related to the release of dartsim 6.14.2

jslee02 commented 4 days ago

@scpeters Would it be beneficial to set up tests on the dartsim/dart repo to identify any issues with the Gazebo integration? Could someone help create a CI job for this in the dartsim/dart repo? Testing building and running Dart and Gazebo-Physics within a Docker environment would suffice.

scpeters commented 4 days ago

@scpeters Would it be beneficial to set up tests on the dartsim/dart repo to identify any issues with the Gazebo integration? Could someone help create a CI job for this in the dartsim/dart repo? Testing building and running Dart and Gazebo-Physics within a Docker environment would suffice.

good idea, I'll discuss with the simulation team

scpeters commented 3 days ago

running UNIT_EntityManagement_TEST on my local system shows invalid data for the link name:

/Users/scpeters/ws/dome/src/ign-physics/dartsim/src/EntityManagement_TEST.cc:88: Failure
Expected equality of these values:
  "empty link"
  link->GetName()
    Which is: "\xC9\xEF\xBF\xF7\x7F\0\0@\x82\xA6\0\x1\0\0\0\x14" "empt"

here's the full backtrace

[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from EntityManagement_TEST
[ RUN      ] EntityManagement_TEST.ConstructEmptyWorld
/Users/scpeters/ws/dome/src/ign-physics/dartsim/src/EntityManagement_TEST.cc:88: Failure
Expected equality of these values:
  "empty link"
  link->GetName()
    Which is: "\xC9\xEF\xBF\xF7\x7F\0\0@\x82\xA6\0\x1\0\0\0\x14" "empt"
Warning [BodyNode.cpp:482] [BodyNode] A negative or zero mass [0] is set to BodyNode [???@??
dumm], which can cause invalid physical behavior or segfault. Consider setting positive value instead.
Process 21694 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x0000000101721713 libdart.6.14.dylib`dart::dynamics::BodyNode::aggregateExternalForces(Eigen::Matrix<double, -1, 1, 0, -1, 1>&) + 211
libdart.6.14.dylib`dart::dynamics::BodyNode::aggregateExternalForces:
->  0x101721713 <+211>: movq   (%rdi), %rax
    0x101721716 <+214>: callq  *0x68(%rax)
    0x101721719 <+217>: testq  %rax, %rax
    0x10172171c <+220>: je     0x10172188f               ; <+591>
Target 0: (UNIT_EntityManagement_TEST) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000101721713 libdart.6.14.dylib`dart::dynamics::BodyNode::aggregateExternalForces(Eigen::Matrix<double, -1, 1, 0, -1, 1>&) + 211
    frame #1: 0x00000001017f71e2 libdart.6.14.dylib`dart::dynamics::Skeleton::addEntryToJointNameMgr(dart::dynamics::Joint*, bool) + 110
    frame #2: 0x0000000101800f89 libdart.6.14.dylib`dart::dynamics::Skeleton::registerJoint(dart::dynamics::Joint*) + 45
    frame #3: 0x00000001017f5cfd libdart.6.14.dylib`dart::dynamics::Skeleton::registerBodyNode(dart::dynamics::BodyNode*) + 1017
    frame #4: 0x000000010180353b libdart.6.14.dylib`dart::dynamics::Skeleton::moveBodyNodeTree(dart::dynamics::Joint*, dart::dynamics::BodyNode*, std::__1::shared_ptr<dart::dynamics::Skeleton>, dart::dynamics::BodyNode*) + 2869
    frame #5: 0x0000000100995108 libgz-physics7-dartsim-plugin.7.dylib`dart::dynamics::RevoluteJoint* dart::dynamics::Skeleton::moveBodyNodeTree<dart::dynamics::RevoluteJoint>(this=0x000000011103a000, _bodyNode=0x000000011103f000, _newSkeleton=std::__1::shared_ptr<dart::dynamics::Skeleton>::element_type @ 0x000000011103a000 strong=11 weak=8, _parentNode=0x0000000000000000, _joint=0x00007ff7bfefd7c0) at Skeleton.hpp:51:7 [opt]
    frame #6: 0x0000000100990ab4 libgz-physics7-dartsim-plugin.7.dylib`dart::dynamics::RevoluteJoint* dart::dynamics::BodyNode::moveTo<dart::dynamics::RevoluteJoint>(this=0x000000011103f000, _newParent=0x0000000000000000, _joint=0x00007ff7bfefd7c0) at BodyNode.hpp:50:36 [opt]
    frame #7: 0x00000001009905d9 libgz-physics7-dartsim-plugin.7.dylib`gz::physics::dartsim::JointFeatures::AttachRevoluteJoint(this=0x0000000111032e00, _childID=0x00007ff7bfefe780, _parent=<unavailable>, _name="revolute", _axis=<unavailable>) at JointFeatures.cc:613:11 [opt]
    frame #8: 0x0000000100990be5 libgz-physics7-dartsim-plugin.7.dylib`virtual thunk to gz::physics::dartsim::JointFeatures::AttachRevoluteJoint(gz::physics::Identity const&, gz::physics::EntityPtr<gz::physics::Link<gz::physics::FeaturePolicy<double, 3ul>, gz::physics::FeatureList<gz::physics::Feature>>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Eigen::Matrix<double, 3, 1, 0, 3, 1> const&) at JointFeatures.cc:0 [opt]
    frame #9: 0x00000001000192f3 UNIT_EntityManagement_TEST`gz::physics::AttachRevoluteJointFeature::Link<gz::physics::FeaturePolicy<double, 3ul>, TestFeatureList>::AttachRevoluteJoint(this=<unavailable>, _parent=0x00007ff7bfefdb00, _name="revolute", _axis=0x00007ff7bfefdcc0) at RevoluteJoint.hh:57:17 [opt]
    frame #10: 0x00000001000101c0 UNIT_EntityManagement_TEST`EntityManagement_TEST_ConstructEmptyWorld_Test::TestBody(this=<unavailable>) at EntityManagement_TEST.cc:94:22 [opt]
    frame #11: 0x0000000100049621 UNIT_EntityManagement_TEST`void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) [inlined] void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(object=<unavailable>, method=0x00000000000000000000000000000000, location="the test body") at gtest.cc:2599:10 [opt]
    frame #12: 0x0000000100049611 UNIT_EntityManagement_TEST`void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(object=0x00006000000094a0, method=0x00000000000000000000000000000000, location="the test body") at gtest.cc:2635:14 [opt]
    frame #13: 0x0000000100049589 UNIT_EntityManagement_TEST`testing::Test::Run(this=0x00006000000094a0) at gtest.cc:2674:5 [opt]
    frame #14: 0x000000010004a4b9 UNIT_EntityManagement_TEST`testing::TestInfo::Run(this=0x000000010dd09370) at gtest.cc:2853:11 [opt]
    frame #15: 0x000000010004b2f7 UNIT_EntityManagement_TEST`testing::TestSuite::Run(this=0x000000010dd096e0) at gtest.cc:3012:30 [opt]
    frame #16: 0x000000010005b067 UNIT_EntityManagement_TEST`testing::internal::UnitTestImpl::RunAllTests(this=0x000000010dd09480) at gtest.cc:5870:44 [opt]
    frame #17: 0x000000010005a471 UNIT_EntityManagement_TEST`bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) [inlined] bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(object=<unavailable>, method=0x00000000000000000000000000000000, location="auxiliary test code (environments or event listeners)") at gtest.cc:2599:10 [opt]
    frame #18: 0x000000010005a461 UNIT_EntityManagement_TEST`bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(object=<unavailable>, method=0x00000000000000000000000000000000, location="auxiliary test code (environments or event listeners)") at gtest.cc:2635:14 [opt]
    frame #19: 0x000000010005a3fd UNIT_EntityManagement_TEST`testing::UnitTest::Run(this=0x00000001000842f0) at gtest.cc:5444:10 [opt]
    frame #20: 0x0000000100068c1a UNIT_EntityManagement_TEST`main [inlined] RUN_ALL_TESTS() at gtest.h:2293:73 [opt]
    frame #21: 0x0000000100068c0d UNIT_EntityManagement_TEST`main(argc=1, argv=0x00007ff7bfeff6c8) at gtest_main.cc:51:10 [opt]
    frame #22: 0x00007ff807acd41f dyld`start + 1903
(lldb)
scpeters commented 1 day ago

This bug has been pretty tricky to track down. I tried building dartsim with gz-physics7 in a colcon workspace on macOS, but that didn't reproduce the problem. The only way I can reproduce it is by installing dartsim with brew install dartsim and compiling gz-physics7 against it.

The diff from 6.13.2 to 6.14.2 is pretty huge (https://github.com/dartsim/dart/compare/v6.13.2...v6.14.2), and seems to have more changes than are listed in the changelog. To investigate further, I've been using git bisect and modifying the dartsim formula to install the recommended commits and then compiling gz-physics7 against it. So far my bisecting has narrowed the possible source of the bug to the following range of git commits: https://github.com/dartsim/dart/compare/383140b17652754523ad53cc5ba2197cc621ba78...d494604ecef25c9ab2c55c38739163a51b0a221b, and I'm suspicious of https://github.com/dartsim/dart/commit/d494604ecef25c9ab2c55c38739163a51b0a221b due to the following comment:

https://github.com/dartsim/dart/blob/v6.14.2/CMakeLists.txt#L80-L88

scpeters commented 1 day ago

I just confirmed that building the homebrew formula with -DDART_ENABLE_SIMD=OFF appears to fix the issue. I'll submit a pull request to homebrew-core

scpeters commented 1 day ago

I just confirmed that building the homebrew formula with -DDART_ENABLE_SIMD=OFF appears to fix the issue. I'll submit a pull request to homebrew-core

https://github.com/Homebrew/homebrew-core/pull/176368

jslee02 commented 1 day ago

Ah, it's my bad. -DDART_ENABLE_SIMD=OFF should be the default. Let me create a fix for the next patch release (6.14.3).

scpeters commented 1 day ago

https://github.com/Homebrew/homebrew-core/pull/176368 has been merged and gz-physics tests are passing again. I'll go ahead and re-trigger CI for the other branches and mark this as fixed

Build Status https://build.osrfoundation.org/job/gz_physics-ci-gz-physics7-homebrew-amd64/64/