Crash with toolchain-2.8

[konradb3]

When build with toolchain-2.8 conman crash (segmentation fault) when switchController is called from ROS. deployer output : 34.172 [ ERROR ][ScriptingService] You're using call() an OwnThread operation or collect() on a sent operation without setting a caller in the OperationCaller. This often causes deadlocks. 34.172 [ ERROR ][ScriptingService] Use this->engine() in a component or GlobalEngine::Instance() in a non-component function. Returning a CollectFailure. 34.172 [ ERROR ][ScriptingService] Unknown exception raised while executing an operation. 34.172 [ ERROR ][Scheme::enableBlock] in scheme: unhandled exception in sent operation. 34.172 [ Debug ][Scheme::enableBlock] in scheme: unhandled exception in sent operation. 34.172 [ ERROR ][Scheme::enableBlock] Could not enable block because the given block is NULL. 34.172 [ ERROR ][Scheme::enableBlock] Could not enable block because the given block is NULL. [ERROR] [1426776125.366205408]: Exception thrown while processing service call: Unable to complete the operation call. The called operation has thrown an exception 34.172 [ ERROR ][Scheme::enableBlock] Could not enable block because the given block is NULL.

[jbohren]

@RoseWall can you take a look at this when you get back?

[RoseWall]

We haven't been able to reproduce the issue, can you provide any more information?

[konradb3]

I can generate the error by running fallowing ops in deployer:

import("rtt_ros")
import("rtt_roscomm")
ros.import("rtt_controller_manager_msgs")
ros.import("conman")
ros.import("conman_ros")
ros.import("conman_blocks")

loadComponent("CM", "conman::Scheme");
setActivityOnCPU("CM", 0.1, 0, ORO_SCHED_OTHER, 3);
CM.loadService("conman_ros");
CM.configure();

loadComponent("test1", "conman_blocks::VectorSum");
addPeer("CM", "test1")
CM.addBlock("test1");

CM.start();

then i run:

rosservice call /controller_manager/switch_controller "start_controllers:
- 'test1'
stop_controllers: []
strictness: 0"

this result in SEGFAULT
backtrace:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffb8813700 (LWP 6238)]
0x00007ffff7b11424 in _M_lower_bound (this=0x901470, __k=..., __y=0x901478, __x=0x87cab0)
    at /usr/include/c++/4.8/bits/stl_tree.h:1156
1156          while (__x != 0)
(gdb) bt
#0  0x00007ffff7b11424 in _M_lower_bound (this=0x901470, __k=..., __y=0x901478, __x=0x87cab0)
    at /usr/include/c++/4.8/bits/stl_tree.h:1156
#1  std::_Rb_tree<std::string, std::pair<std::string const, RTT::TaskContext*>, std::_Select1st<std::pair<std::string const, RTT::TaskContext*> >, std::less<std::string>, std::allocator<std::pair<std::string const, RTT::TaskContext*> > >::find (this=this@entry=0x901470, __k=...)
    at /usr/include/c++/4.8/bits/stl_tree.h:1805
#2  0x00007ffff7aff7c1 in count (__x=..., this=0x901470) at /usr/include/c++/4.8/bits/stl_map.h:849
#3  RTT::TaskContext::hasPeer (this=0x901450, peer_name=...)
    at /home/konradb3/ws_conman/src/orocos_toolchain/rtt/rtt/TaskContext.cpp:333
#4  0x00007ffff7aff7e5 in RTT::TaskContext::getPeer (this=0x901450, peer_name=...)
    at /home/konradb3/ws_conman/src/orocos_toolchain/rtt/rtt/TaskContext.cpp:338
#5  0x00007fffbd946e56 in conman::Scheme::disableBlock (this=0x901450, block_name=...)
    at /home/konradb3/ws_conman/src/conman/conman/src/scheme.cpp:1606
#6  0x00007fffbd947616 in conman::Scheme::switchBlocks (this=0x901450, disable_block_names=..., 
    enable_block_names=..., strict=false, force=false)
    at /home/konradb3/ws_conman/src/conman/conman/src/scheme.cpp:1778
#7  0x00007fffbd9857a6 in boost::_mfi::mf4<bool, conman::Scheme, std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, bool, bool>::operator() (this=0x7fffb8812d08, p=0x901450, a1=..., a2=..., a3=false, a4=false)
    at /usr/include/boost/bind/mem_fn_template.hpp:506
#8  0x00007fffbd9841b7 in boost::_bi::list5<boost::_bi::value<conman::Scheme*>, boost::arg<1>, boost::arg<2>, boost::arg<3>, boost::arg<4> >::operator()<bool, boost::_mfi::mf4<bool, conman::Scheme, std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, bool, bool>, boost::_bi::list4<std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, bool&, bool&> > (this=0x7fffb8812d18, f=..., 
    a=...) at /usr/include/boost/bind/bind.hpp:515
#9  0x00007fffbd98321c in boost::_bi::bind_t<boost::_bi::unspecified, boost::_mfi::mf4<bool, conman::Scheme, std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, bool, bool>, boost::_bi::list5<boost::_bi::value<conman::Scheme*>, boost::arg<1>, boost::arg<2>, boost::arg<3>, boost::arg<4> > >::operator()<std::vector<std::string, std::allocator<std::string> > const, std::vector<std::string, std::allocator<std::string> > const, bool, bool> (
    this=0x7fffb8812d08, a1=..., a2=..., a3=@0x7fffb8812b74: false, a4=@0x7fffb8812b70: false)
    at /usr/include/boost/bind/bind_template.hpp:145
#10 0x00007fffbd982271 in boost::detail::function::function_obj_invoker4<boost::_bi::bind_t<boost::_bi::unspecified, boost::_mfi::mf4<bool, conman::Scheme, std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, bool, bool>, boost::_bi::list5<boost::_bi::value<conman::Scheme*>, boost::arg<1>, boost::arg<2>, boost::arg<3>, boost::arg<4> > >, bool, std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, bool, bool>::invoke (function_obj_ptr=..., a0=..., a1=..., a2=false, a3=false)
    at /usr/include/boost/function/function_template.hpp:132
#11 0x00007fffbd9a59fb in boost::function4<bool, std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, bool, bool>::operator() (
    this=0x7fffb8812d00, a0=..., a1=..., a2=false, a3=false)
    at /usr/include/boost/function/function_template.hpp:767
#12 0x00007fffbd9b81fa in boost::_bi::list4<boost::reference_wrapper<std::vector<std::string, std::allocator<std::string> > const>, boost::reference_wrapper<std::vector<std::string, std::allocator<std::string> > const>, boost::reference_wrapper<bool>, boost::reference_wrapper<bool> >::operator()<bool, boost::function<bool (std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, bool, bool)>, boost::_bi::list0>(boost::_bi::type<bool>, boost::function<bool (std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, bool, bool)>&, boost::_bi::list0&, long) (this=0x7fffb8812d20, f=..., a=...)
    at /usr/include/boost/bind/bind.hpp:447
#13 0x00007fffbd9af67f in boost::_bi::bind_t<boost::_bi::unspecified, boost::function<bool (std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, bool, bool)>, boost::_bi::list4<boost::reference_wrapper<std::vector<std::string, std::allocator<std::string> > const>, boost::reference_wrapper<std::vector<std::string, std::allocator<std::string> > const>, boost::reference_wrapper<bool>, boost::reference_wrapper<bool> > >::operator()() (this=0x7fffb8812d00)
    at /usr/include/boost/bind/bind_template.hpp:20
#14 0x00007fffbd9a5e43 in RTT::internal::RStore<bool>::exec<boost::_bi::bind_t<boost::_bi::unspecified, boost::function<bool (std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, bool, bool)>, boost::_bi::list4<boost::reference_wrapper<std::vector<std::string, std::allocator<std::string> > const>, boost::reference_wrapper<std::vector<std::string, st---Type <return> to continue, or q <return> to quit---
d::allocator<std::string> > const>, boost::reference_wrapper<bool>, boost::reference_wrapper<bool> > > >(boost::_bi::bind_t<boost::_bi::unspecified, boost::function<bool (std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, bool, bool)>, boost::_bi::list4<boost::reference_wrapper<std::vector<std::string, std::allocator<std::string> > const>, boost::reference_wrapper<std::vector<std::string, std::allocator<std::string> > const>, boost::reference_wrapper<bool>, boost::reference_wrapper<bool> > >) (this=0x7ffff7f2f97a, f=...)
    at /home/konradb3/ws_conman/install/include/rtt/internal/BindStorage.hpp:155
#15 0x00007fffbd9999e4 in RTT::internal::BindStorageImpl<4, bool (std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, bool, bool)>::exec() (this=0x7ffff7f2f948) at /home/konradb3/ws_conman/install/include/rtt/internal/BindStorage.hpp:466
#16 0x00007fffbd98e8ad in RTT::internal::LocalOperationCallerImpl<bool (std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&, bool, bool)>::executeAndDispose() (this=0x7ffff7f2f910)
    at /home/konradb3/ws_conman/install/include/rtt/internal/../internal/LocalOperationCaller.hpp:101
#17 0x00007ffff7afc88a in RTT::ExecutionEngine::processMessages (this=0x8cbe50)
    at /home/konradb3/ws_conman/src/orocos_toolchain/rtt/rtt/ExecutionEngine.cpp:229
#18 0x00007ffff7afcf49 in RTT::ExecutionEngine::step (this=0x8cbe50)
    at /home/konradb3/ws_conman/src/orocos_toolchain/rtt/rtt/ExecutionEngine.cpp:352
#19 0x00007ffff7b2d76c in RTT::os::thread_function (t=0x96c420)
    at /home/konradb3/ws_conman/src/orocos_toolchain/rtt/rtt/os/Thread.cpp:100
#20 0x00007ffff7b3112a in RTT::os::rtos_posix_thread_wrapper (cookie=0x87a550)
    at /home/konradb3/ws_conman/src/orocos_toolchain/rtt/rtt/os/gnulinux/fosi_internal.cpp:98
#21 0x00007ffff6a81182 in start_thread (arg=0x7fffb8813700) at pthread_create.c:312
#22 0x00007ffff629447d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

[konradb3]

ping

[jbohren]

We've found a fix for this, and @RoseWall is applying it to the other cases before creating a PR.

[konradb3]

The #31 fix the crash but some warnings remains: 3.248 [ Warning][ScriptingService] Extra argument discarded for OperationCallerC. 3.248 [ Warning][ScriptingService] Extra argument discarded for OperationCallerC. 3.248 [ Warning][ScriptingService] Extra argument discarded for OperationCallerC. 3.248 [ Warning][ScriptingService] Extra argument discarded for OperationCallerC. 3.249 [ ERROR ][ScriptingService] Constructing RemoteOperationCaller for getBlocks was not possible. 3.249 [ ERROR ][ScriptingService] Probable cause: Bad DataSource assignment: incompatible types. 3.249 [ Warning][ScriptingService] Extra argument discarded for OperationCallerC. 3.249 [ Warning][ScriptingService] Extra argument discarded for OperationCallerC. 3.249 [ Warning][ScriptingService] Extra argument discarded for OperationCallerC. 3.249 [ Warning][ScriptingService] Extra argument discarded for OperationCallerC. 3.249 [ ERROR ][ScriptingService] Constructing RemoteOperationCaller for getGroups was not possible. 3.249 [ ERROR ][ScriptingService] Probable cause: Bad DataSource assignment: incompatible types.

[jbohren]

@konradb3 Take a look at 4eb0e4a

[konradb3]

It's beater but:

.353 [ Warning][ScriptingService] Extra argument discarded for OperationCallerC. 1.353 [ ERROR ][ScriptingService] Constructing RemoteOperationCaller for getBlocks was not possible. 1.353 [ ERROR ][ScriptingService] Probable cause: Bad DataSource assignment: incompatible types. 1.353 [ ERROR ][ScriptingService] Constructing RemoteOperationCaller for getGroups was not possible. 1.353 [ ERROR ][ScriptingService] Probable cause: Bad DataSource assignment: incompatible types.