ros_datacentre to mongodb_store issue

[ToMadoRe]

@cburbridge: Since I updated ros_datacentre to mongodb_store, I get the following error:

  File "/home/thomas/Projects/strands_catkin_ws/src/strands_tabletop_perception/scene_recorder/scripts/replay_snapshots.py", line 50, in <module>
    tf =  TransformationStore.msg_to_transformer(observation.get_message("/tf"))
  File "/home/thomas/Projects/strands_catkin_ws/src/strands_tabletop_perception/world_state/src/world_state/observation.py", line 142, in get_message
    return self._messages[topic].retrieve()
  File "/home/thomas/Projects/strands_catkin_ws/src/strands_tabletop_perception/world_state/src/world_state/observation.py", line 88, in retrieve
    return proxy.query_id(self.obj_id, self.typ)[0]
  File "/home/thomas/Projects/strands_catkin_ws/src/mongodb_store/mongodb_store/src/mongodb_store/message_store.py", line 106, in query_id
    return self.query(type, {'_id': ObjectId(id)}, {}, True)
  File "/home/thomas/Projects/strands_catkin_ws/src/mongodb_store/mongodb_store/src/mongodb_store/message_store.py", line 225, in query
    response = self.query_id_srv(self.database, self.collection, type, single, StringPairList(message_tuple), StringPairList(meta_tuple))       
  File "/opt/ros/hydro/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 435, in __call__
    return self.call(*args, **kwds)
  File "/opt/ros/hydro/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 515, in call
    responses = transport.receive_once()
  File "/opt/ros/hydro/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 667, in receive_once
    p.read_messages(b, msg_queue, sock) 
  File "/opt/ros/hydro/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 353, in read_messages
    self._read_ok_byte(b, sock)
  File "/opt/ros/hydro/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 336, in _read_ok_byte
    raise ServiceException("service [%s] responded with an error: %s"%(self.resolved_name, str))
rospy.service.ServiceException: service [/message_store/query_messages] responded with an error: error processing request: No module named ros_datacentre_msgs.msg._SerialisedMessage

Do you (or anyone else) know what causes this?

[cburbridge]

Do you get the same issue if you build the workspace fresh, having removed the non src/ folders? @hawesie might know what is wrong..

On 18 August 2014 16:22, Thomas Fäulhammer notifications@github.com wrote:

@cburbridge https://github.com/cburbridge: Since I updated ros_datacentre to mongodb_store, I get the following error:

File "/home/thomas/Projects/strands_catkin_ws/src/strands_tabletop_perception/scene_recorder/scripts/replay_snapshots.py", line 50, in tf = TransformationStore.msg_to_transformer(observation.get_message("/tf")) File "/home/thomas/Projects/strands_catkin_ws/src/strands_tabletop_perception/world_state/src/world_state/observation.py", line 142, in get_message return self._messages[topic].retrieve() File "/home/thomas/Projects/strands_catkin_ws/src/strands_tabletop_perception/world_state/src/world_state/observation.py", line 88, in retrieve return proxy.query_id(self.obj_id, self.typ)[0] File "/home/thomas/Projects/strands_catkin_ws/src/mongodb_store/mongodb_store/src/mongodb_store/message_store.py", line 106, in query_id return self.query(type, {'_id': ObjectId(id)}, {}, True) File "/home/thomas/Projects/strands_catkin_ws/src/mongodb_store/mongodb_store/src/mongodb_store/message_store.py", line 225, in query response = self.query_id_srv(self.database, self.collection, type, single, StringPairList(message_tuple), StringPairList(meta_tuple)) File "/opt/ros/hydro/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 435, in call return self.call(_args, *_kwds) File "/opt/ros/hydro/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 515, in call responses = transport.receive_once() File "/opt/ros/hydro/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 667, in receive_once p.read_messages(b, msg_queue, sock) File "/opt/ros/hydro/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 353, in read_messages self._read_ok_byte(b, sock) File "/opt/ros/hydro/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 336, in _read_ok_byte raise ServiceException("service [%s] responded with an error: %s"%(self.resolved_name, str)) rospy.service.ServiceException: service [/message_store/query_messages] responded with an error: error processing request: No module named ros_datacentre_msgs.msg._SerialisedMessage

Do you (or anyone else) know what causes this?

— Reply to this email directly or view it on GitHub https://github.com/strands-project/strands_tabletop_perception/issues/76 .

[ToMadoRe]

Yes, I removed the /build and /devel folder before building it. I can't even find anything with "grep -r ros_datacentre *" in the strands_catkin_ws, which is a bit strange considering the error message.

[ToMadoRe]

There is one more thing: When I start mongodb (roslaunch mongodb_store mongodb_store.launch db_path:=/opt/strands/ros_datacentre db_port:=62345), the config manager crashes:

[config_manager-2] process has died [pid 6621, exit code 1, cmd /home/thomas/Projects/strands_catkin_ws/src/mongodb_store/mongodb_store/scripts/config_manager.py __name:=config_manager __log:=/home/thomas/.ros/log/84b9a20e-2713-11e4-8564-3ca9f423a5c0/config_manager-2.log].
log file: /home/thomas/.ros/log/84b9a20e-2713-11e4-8564-3ca9f423a5c0/config_manager-2*.log

Don't know if this is related to the issue.

[cburbridge]

Ok, the problem is a problem. Messages of the type ros_datacentre_msgs/* that have been stored in the datacentre using the old message store can't be recreated now with the new message store since that message definition does not exist anywhere. In the world_state package, I use ros_datacentre_msgs/SerialisedMessage to store serialised transforms from /tf.

Probably the best solution is to scan through every entry in every collection in the message_store database for stored objects with field "_meta.stored_type" containing ros_datacentre_msgs/* and chaning it to mongo_store_msgs/*. Alternatively, create a ros_datacentre_msgs package mirroring mongodb_store_msgs.

But this does not explain the problem with the config manager. Does it output any error message?

On 18 August 2014 21:13, Thomas Fäulhammer notifications@github.com wrote:

There is one more thing: When I start mongodb (roslaunch mongodb_store mongodb_store.launch db_path:=/opt/strands/ros_datacentre db_port:=62345), the config manager crashes:

[config_manager-2] process has died [pid 6621, exit code 1, cmd /home/thomas/Projects/strands_catkin_ws/src/mongodb_store/mongodb_store/scripts/config_manager.py __name:=config_manager __log:=/home/thomas/.ros/log/84b9a20e-2713-11e4-8564-3ca9f423a5c0/config_manager-2.log]. log file: /home/thomas/.ros/log/84b9a20e-2713-11e4-8564-3ca9f423a5c0/config_manager-2*.log

Don't know if this is related to the issue.

— Reply to this email directly or view it on GitHub.

[ToMadoRe]

I see. Couldn't find out how to the mongodb query on "_meta.stored_type" looks like, so I just cloned the msgs package. Now it complains about the datacentre_host:

  File "/home/thomas/Projects/strands_catkin_ws/src/strands_tabletop_perception/scene_recorder/scripts/replay_snapshots.py", line 21, in <module>
    mongodb =  MongoConnection(database_name="snapshots")
  File "/home/thomas/Projects/strands_catkin_ws/src/strands_tabletop_perception/world_state/src/world_state/mongo.py", line 27, in __init__
    server = rospy.get_param("datacentre_host")
  File "/opt/ros/hydro/lib/python2.7/dist-packages/rospy/client.py", line 452, in get_param
    return _param_server[param_name] #MasterProxy does all the magic for us
  File "/opt/ros/hydro/lib/python2.7/dist-packages/rospy/msproxy.py", line 117, in __getitem__
    raise KeyError(key)
KeyError: 'datacentre_host'

Do I have to set that somewhere?

Regarding config_manager: No, there is just the output from the launch file:

[config_manager-2] process has died [pid 14569, exit code 1, cmd /home/thomas/Projects/strands_catkin_ws/src/mongodb_store/mongodb_store/scripts/config_manager.py __name:=config_manager __log:=/home/thomas/.ros/log/3fcec730-2724-11e4-b82f-3ca9f423a5c0/config_manager-2.log].
log file: /home/thomas/.ros/log/3fcec730-2724-11e4-b82f-3ca9f423a5c0/config_manager-2*.log

If I start the config_manager alone (without any arguments), then there is:

$ rosrun mongodb_store config_manager.py 
Traceback (most recent call last):
  File "/home/thomas/Projects/strands_catkin_ws/src/mongodb_store/mongodb_store/scripts/config_manager.py", line 24, in <module>
    class MongoTransformer(pymongo.son_manipulator.SONManipulator):
AttributeError: 'module' object has no attribute 'son_manipulator'
thomas@romeo:~/Projects/strands_catkin_ws$ rosrun mongodb_store config_manager.py default_path:=
Traceback (most recent call last):
  File "/home/thomas/Projects/strands_catkin_ws/src/mongodb_store/mongodb_store/scripts/config_manager.py", line 24, in <module>
    class MongoTransformer(pymongo.son_manipulator.SONManipulator):
AttributeError: 'module' object has no attribute 'son_manipulator'

[hawesie]

Please search and replace datacentre_ with mongodb_ in all your files. I renamed these as part of the ros_datacentre renaming, but didn’t fix all the STRANDS packages that depend on it.

[cburbridge]

Has the config manager always crashed on your system? What version of pymongo do you run?

On 19 August 2014 08:34, Nick Hawes notifications@github.com wrote:

Please search and replace datacentre_ with mongodb_ in all your files. I renamed these as part of the ros_datacentre renaming, but didn’t fix all the STRANDS packages that depend on it.

— Reply to this email directly or view it on GitHub https://github.com/strands-project/strands_tabletop_perception/issues/76#issuecomment-52599593 .

[ToMadoRe]

I just did a pymongo upgrade, so I should have pymongo 2.7.2 Still the same error:

Traceback (most recent call last):
  File "/home/thomas/Projects/strands_catkin_ws/src/mongodb_store/mongodb_store/scripts/config_manager.py", line 24, in <module>
    class MongoTransformer(pymongo.son_manipulator.SONManipulator):
AttributeError: 'module' object has no attribute 'son_manipulator'
process[message_store-3]: started with pid [21262]
[config_manager-2] process has died [pid 21241, exit code 1, cmd /home/thomas/Projects/strands_catkin_ws/src/mongodb_store/mongodb_store/scripts/config_manager.py __name:=config_manager __log:=/home/thomas/.ros/log/caeabbec-2771-11e4-8ced-3c970e8d21f3/config_manager-2.log].
log file: /home/thomas/.ros/log/caeabbec-2771-11e4-8ced-3c970e8d21f3/config_manager-2*.log

[cburbridge]

Please try this PR: https://github.com/strands-project/mongodb_store/pull/79.

[ToMadoRe]

Works :+1: