search

gazebosim / gazebo-classic

Gazebo classic. For the latest version, see https://github.com/gazebosim/gz-sim
http://classic.gazebosim.org/
Other
1.17k stars 477 forks source link

[build cop] PhysicsEngines/HeightmapTest.MaterialShader broken on OSX #2260

Open osrf-migration opened 7 years ago

osrf-migration commented 7 years ago

Original report (archived issue) by Shane Loretz (Bitbucket: Shane Loretz, GitHub: sloretz).

Test failures were seen on gazebo-ci-gazebo8-homebrew-amd64 job 30. Creating this issue to track a possible flaky test.

523: [ RUN      ] PhysicsEngines/HeightmapTest.MaterialShader/0
523: [Msg] Waiting for master.
523: [Msg] Connected to gazebo master @ http://127.0.0.1:11345
523: [Msg] Publicized address: 172.23.2.252
523: [Dbg] [ServerFixture.cc:209] ServerFixture load in 0.9 seconds, timeout after 600 seconds
523: [Msg] Loading heightmap: heightmap_bowl
523: [Msg] Loading heightmap cache data: /Users/jenkins/.gazebo/paging/heightmap_bowl/gazebo_terrain_00000000.dat
523: [Msg] Heightmap loaded. Process took: 0.054284 seconds
523: /Users/jenkins/workspace/gazebo-ci-gazebo8-homebrew-amd64/gazebo8/test/integration/heightmap.cc:515: Failure
523: Expected: (rSum) > (gSum), actual: 0 vs 0
523: /Users/jenkins/workspace/gazebo-ci-gazebo8-homebrew-amd64/gazebo8/test/integration/heightmap.cc:516: Failure
523: Expected: (rSum) > (bSum), actual: 0 vs 0
523: [Dbg] [ServerFixture.cc:135] ServerFixture::Unload
523: [  FAILED  ] PhysicsEngines/HeightmapTest.MaterialShader/0, where GetParam() = "ode" (2226 ms)
523: [ RUN      ] PhysicsEngines/HeightmapTest.MaterialShader/1
523: [Msg] Waiting for master.
523: [Msg] Connected to gazebo master @ http://127.0.0.1:11345
523: [Msg] Publicized address: 172.23.2.252
523: [Msg] Loading heightmap: heightmap_bowl
523: [Msg] Loading heightmap cache data: /Users/jenkins/.gazebo/paging/heightmap_bowl/gazebo_terrain_00000000.dat
523: [Dbg] [ServerFixture.cc:209] ServerFixture load in 0.7 seconds, timeout after 600 seconds
523: [Msg] Heightmap loaded. Process took: 0.05873 seconds
523: /Users/jenkins/workspace/gazebo-ci-gazebo8-homebrew-amd64/gazebo8/test/integration/heightmap.cc:515: Failure
523: Expected: (rSum) > (gSum), actual: 0 vs 0
523: /Users/jenkins/workspace/gazebo-ci-gazebo8-homebrew-amd64/gazebo8/test/integration/heightmap.cc:516: Failure
523: Expected: (rSum) > (bSum), actual: 0 vs 0
523: [Dbg] [ServerFixture.cc:135] ServerFixture::Unload
523: [  FAILED  ] PhysicsEngines/HeightmapTest.MaterialShader/1, where GetParam() = "bullet" (2174 ms)
523: [ RUN      ] PhysicsEngines/HeightmapTest.MaterialShader/2
523: [Msg] Waiting for master.
523: [Msg] Connected to gazebo master @ http://127.0.0.1:11345
523: [Msg] Publicized address: 172.23.2.252
523: [Err] [SimbodyHeightmapShape.cc:33] SimbodyHeightmapShape not implemented yet.
523: [Err] [SimbodyPhysics.cc:1534] Collision type [589824] unimplemented
523: [Msg] Loading heightmap: heightmap_bowl
523: [Msg] Loading heightmap cache data: /Users/jenkins/.gazebo/paging/heightmap_bowl/gazebo_terrain_00000000.dat
523: [Dbg] [ServerFixture.cc:209] ServerFixture load in 0.9 seconds, timeout after 600 seconds
523: [Msg] Heightmap loaded. Process took: 0.05474 seconds
523: /Users/jenkins/workspace/gazebo-ci-gazebo8-homebrew-amd64/gazebo8/test/integration/heightmap.cc:515: Failure
523: Expected: (rSum) > (gSum), actual: 1738380 vs 1738380
523: /Users/jenkins/workspace/gazebo-ci-gazebo8-homebrew-amd64/gazebo8/test/integration/heightmap.cc:516: Failure
523: Expected: (rSum) > (bSum), actual: 1738380 vs 1738380
523: [Dbg] [ServerFixture.cc:135] ServerFixture::Unload
523: [  FAILED  ] PhysicsEngines/HeightmapTest.MaterialShader/2, where GetParam() = "simbody" (2394 ms)
osrf-migration commented 7 years ago

Original comment by Shane Loretz (Bitbucket: Shane Loretz, GitHub: sloretz).

Maybe a regression rather than a flaky test. It also failed on gazebo-ci-default-homebrew-amd64 job 227

osrf-migration commented 7 years ago

Original comment by Shane Loretz (Bitbucket: Shane Loretz, GitHub: sloretz).

osrf-migration commented 7 years ago

Original comment by Shane Loretz (Bitbucket: Shane Loretz, GitHub: sloretz).

osrf-migration commented 7 years ago

Original comment by Shane Loretz (Bitbucket: Shane Loretz, GitHub: sloretz).

osrf-migration commented 7 years ago

Original comment by Shane Loretz (Bitbucket: Shane Loretz, GitHub: sloretz).

A failure of this test on gazebo-ci-gazebo7-homebrew-amd64 build 143. This time it segfaults

443: [ RUN      ] PhysicsEngines/HeightmapTest.MaterialShader/0
443: [Msg] Waiting for master.
443: [Msg] Connected to gazebo master @ http://127.0.0.1:11345
443: [Msg] Publicized address: 172.23.2.252
443: [Msg] Loading heightmap: heightmap_bowl
443: [Msg] Loading heightmap cache data: /Users/jenkins/.gazebo/paging/heightmap_bowl/gazebo_terrain_00000000.dat
443: [Dbg] [ServerFixture.cc:207] ServerFixture load in 1.1 seconds, timeout after 600 seconds
443: [Msg] Heightmap loaded. Process took: 0.050975 seconds
443: /Users/jenkins/workspace/gazebo-ci-gazebo7-homebrew-amd64/gazebo7/gazebo/test/ServerFixture.cc:279: Failure
443: Expected: this->server->Run() doesn't throw an exception.
443:   Actual: it throws.
443: /Users/jenkins/workspace/gazebo-ci-gazebo7-homebrew-amd64/gazebo7/gazebo/test/ServerFixture.cc:1318: Failure
443: Expected: (i) < (_retries), actual: 100 vs 100
443: /Users/jenkins/workspace/gazebo-ci-gazebo7-homebrew-amd64/gazebo7/gazebo/test/ServerFixture.cc:1321: Failure
443: Failed
443: ServerFixture timeout: max number of retries (100) exceeded while awaiting the spawn of camera_sensor
1/2 Test #443: INTEGRATION_heightmap ............***Exception: SegFault 54.74 sec
test 444
    Start 444: check_INTEGRATION_heightmap

444: Test command: /Users/jenkins/workspace/gazebo-ci-gazebo7-homebrew-amd64/gazebo7/tools/check_test_ran.py "/Users/jenkins/workspace/gazebo-ci-gazebo7-homebrew-amd64/build/test_results/INTEGRATION_heightmap.xml"
444: Test timeout computed to be: 9.99988e+06
444: Checking for test results in /Users/jenkins/workspace/gazebo-ci-gazebo7-homebrew-amd64/build/test_results/INTEGRATION_heightmap.xml
444: Cannot find results, writing failure results to /Users/jenkins/workspace/gazebo-ci-gazebo7-homebrew-amd64/build/test_results/INTEGRATION_heightmap.xml
2/2 Test #444: check_INTEGRATION_heightmap ......***Failed    0.03 sec

0% tests passed, 2 tests failed out of 2
osrf-migration commented 7 years ago

Original comment by Steve Peters (Bitbucket: Steven Peters, GitHub: scpeters).

On homebrew, we added an ogre1.9 formula for gazebo8, but the gazebo7 formula still uses the ogre formula (version 1.7.4). I recently noticed on my laptop that I would have to clear the ~/.gazebo/paging folder after switching between ogre versions in order for the textures to load properly. I'll try clearing that folder as part of the CI build script and see if it fixes this problem.

We can also switch the gazebo7 formula to ogre1.9 to see if that helps.

osrf-migration commented 7 years ago

Original comment by Steve Peters (Bitbucket: Steven Peters, GitHub: scpeters).

Trying the idea of clearing the paging folder during homebrew CI builds:

osrf-migration commented 7 years ago

Original comment by Ian Chen (Bitbucket: Ian Chen, GitHub: iche033).

thanks for looking into this. I think we may need to downgrade the shaders used in this test, i.e. heightmap_custom_fp.glsl and heightmap_custom_vp.glsl from glsl version 130 to 120 for it to work on OSX. I'm surprised that it segfaulted. I expected that imcompatible glsl version would just result in an error or exception message in ogre.log and the heightmap would still render but with a white material.