Race condition in ModelRanger::Sensor

richmattes commented 7 years ago

This bug came out of a thread on the player-users mailing list.

Software Version

Stage version: 4.3.0 Player version: 3.1.0

Bug Description

I believe there's a race condition between the worker thread that runs ModelRanger::Sensor::Update and the main thread that is responsible for visualization (and calls ModelRanger::Sensor::Visualize) The segmentation fault occurs in ModelRanger::Sensor::Visualize, where the code tries to access the ranges member vector. At this point, the ranges vector size is zero, so accessing the first index of the vector causes a segfault. A gdb trace is supplied at the end of the bug report.

The ranges vector gets resized to the proper size in the ModelRanger::Sensor::Update function, so I believe that the visualize function is getting called before the update function.

Possible solutions:

Ensure Update always gets called before Visualize
Initialize the member variables earler, such as in the Load function
Add a check to the Visualize function that returns early if ranges.size() is 0

How to reproduce

Edit simple.world to disable controllers, and to set the speedup to 1
Run player simple.cfg
Run a player client (e.g. playerprint ranger)

Expected Results

The ranger values get printed through the player proxy.

Actual results

Segmentation Fault

GDB output

(gdb) bt
#0  Stg::ModelRanger::Sensor::Visualize (this=this@entry=0x103c46220, vis=vis@entry=0x103a0ea98, 
    rgr=rgr@entry=0x103a0e6f0) at /usr/src/debug/Stage-4.3.0/libstage/model_ranger.cc:327
#1  0x00007f0d50a30aa5 in Stg::ModelRanger::Vis::Visualize (this=0x103a0ea98, mod=<optimized out>, 
    cam=<optimized out>) at /usr/src/debug/Stage-4.3.0/libstage/model_ranger.cc:428
#2  0x00007f0d50a2895a in Stg::Model::DataVisualizeTree (this=0x103a0e6f0, cam=cam@entry=0x1004c3fc8)
    at /usr/src/debug/Stage-4.3.0/libstage/model_draw.cc:539
#3  0x00007f0d50a2898f in Stg::Model::DataVisualizeTree (this=<optimized out>, cam=0x1004c3fc8)
    at /usr/src/debug/Stage-4.3.0/libstage/model_draw.cc:545
#4  0x00007f0d50a490cc in Stg::Canvas::renderFrame (this=0x1004c3e40)
    at /usr/src/debug/Stage-4.3.0/libstage/canvas.cc:928
#5  0x00007f0d50a46dae in Stg::Canvas::draw (this=0x1004c3e40)
    at /usr/src/debug/Stage-4.3.0/libstage/canvas.cc:1251
#6  0x00007f0d5015a156 in Fl_Gl_Window::flush (this=0x1004c3e40) at Fl_Gl_Window.cxx:392
#7  0x00007f0d4fe76667 in Fl_X::flush (this=0x1005a9330) at ../FL/x.H:150
#8  Fl::flush () at Fl.cxx:788
#9  0x00007f0d4fe7779d in Fl::wait (time_to_wait=0.01961199999999998, time_to_wait@entry=1e+20)
    at Fl.cxx:604
#10 0x00007f0d4fe7786d in Fl::wait () at Fl.cxx:652
#11 0x00007f0d50c71085 in StgDriver::Update (this=0x100428c60)
    at /usr/src/debug/Stage-4.3.0/libstageplugin/p_driver.cc:607
#12 0x00007f0d7c39e729 in DeviceTable::UpdateDevices (this=<optimized out>)
    at /usr/src/debug/player-release-3-1-0/libplayercore/devicetable.cc:255
#13 0x00000001000031b0 in main (argc=<optimized out>, argv=<optimized out>)
    at /usr/src/debug/player-release-3-1-0/server/server.cc:443
(gdb) list
322   
323   std::vector<glpoint_t> verts( sample_count );
324   
325   if (sample_count == 1) {
326     // only one sample, so we fake up some beam width for beauty
327     const double sidelen = ranges[0];
328     const double da = fov / 2.0;
329     
330     verts.resize(3);
331     verts[0].x = 0;
(gdb) p ranges.size()
$1 = 0

NippunKumaar commented 6 years ago

Is this problem solved? I am facing this issue. Is there a way to overcome this with our controller code? I am currently using Stage 4.3.0 Player 3.1.0 Ubuntu 16.04 LTS

themultiplexer commented 3 years ago

I am experiencing the same issue on Ubuntu 18.04 (NVidia Grapics) with melodic.

ros-melodic-stage 4.3.0 ros-melodic-stage-ros 1.8.0

Thread 1 "stageros" received signal SIGSEGV, Segmentation fault.
0x00007ffff693f0ca in Stg::ModelRanger::Sensor::Visualize(Stg::ModelRanger::Vis*, Stg::ModelRanger*) const () from /opt/ros/melodic/lib/libstage.so.4.3.0
(gdb) bt
#0  0x00007ffff693f0ca in Stg::ModelRanger::Sensor::Visualize(Stg::ModelRanger::Vis*, Stg::ModelRanger*) const () from /opt/ros/melodic/lib/libstage.so.4.3.0
#1  0x00007ffff693f7f5 in Stg::ModelRanger::Vis::Visualize(Stg::Model*, Stg::Camera*) () from /opt/ros/melodic/lib/libstage.so.4.3.0
#2  0x00007ffff6936b8a in Stg::Model::DataVisualizeTree(Stg::Camera*) () from /opt/ros/melodic/lib/libstage.so.4.3.0
#3  0x00007ffff6936bbf in Stg::Model::DataVisualizeTree(Stg::Camera*) () from /opt/ros/melodic/lib/libstage.so.4.3.0
#4  0x00007ffff6956d3c in Stg::Canvas::renderFrame() () from /opt/ros/melodic/lib/libstage.so.4.3.0
#5  0x00007ffff6954a4e in Stg::Canvas::draw() () from /opt/ros/melodic/lib/libstage.so.4.3.0
#6  0x00007ffff34559b6 in Fl_Gl_Window::flush() () from /usr/lib/x86_64-linux-gnu/libfltk_gl.so.1.3
#7  0x00007ffff3164837 in Fl::flush() () from /usr/lib/x86_64-linux-gnu/libfltk.so.1.3
#8  0x00007ffff316594d in Fl::wait(double) () from /usr/lib/x86_64-linux-gnu/libfltk.so.1.3
#9  0x00007ffff3165a1d in Fl::wait() () from /usr/lib/x86_64-linux-gnu/libfltk.so.1.3
#10 0x00007ffff6947941 in Stg::World::Run() () from /opt/ros/melodic/lib/libstage.so.4.3.0
#11 0x0000555555565c83 in main ()

My crappy workaround is a shell script which launches stageros until it doesnt crash :joy: Takes about 15 times ^^

rtv / Stage