strands-project / mongodb_store

MongoDB tools for storing and analysing runs of ROS systems.
BSD 3-Clause "New" or "Revised" License
49 stars 72 forks source link

mongo_server is not properly killed if bind_to_host is set to true #265

Open heuristicus opened 3 years ago

heuristicus commented 3 years ago

When using a basic setup, sending C-c to the terminal window will kill mongo properly

roslaunch mongodb_store mongodb_store.launch db_path:=/home/db
...
^C
...
ps ax | grep mongo
 7487 pts/3    R+     0:00 grep --color=auto mongo

If using bind_to_host, this is not the case.

HOSTNAME=192.168.1.5 roslaunch mongodb_store mongodb_store.launch db_path:=/home/db bind_to_host:=true
...
^C
...
[mongo_server-1] escalating to SIGTERM
[mongo_server-1] escalating to SIGKILL
Shutdown errors:
 * process[mongo_server-1, pid 7503]: required SIGKILL. May still be running.
ps ax | grep mongo
 7533 ?        Sl     0:01 mongod --dbpath /home/db --port 62345 --smallfiles --bind_ip 192.168.1.5
 7610 pts/3    S+     0:00 grep --color=auto mongo

This means that when you next try to start the launch file, you will get something like

[ERROR] [1602235526.494954]: MongoDB process stopped!
[ERROR] [1602235526.495783]: Mongo process error! Exit code=100
[WARN] [1602235526.497455]: It looks like Mongo already died. Watch out as the DB might need recovery time at next run.
Traceback (most recent call last):
  File "/opt/ros/melodic/lib/mongodb_store/message_store_node.py", line 316, in <module>
    store = MessageStore()
  File "/opt/ros/melodic/lib/mongodb_store/message_store_node.py", line 51, in __init__
    have_dc = dc_util.wait_for_mongo(local_timeout)
  File "/opt/ros/melodic/lib/python2.7/dist-packages/mongodb_store/util.py", line 67, in wait_for_mongo
    wait()
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 442, in __call__
    return self.call(*args, **kwds)
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 532, in call
    raise ServiceException("transport error completing service call: %s"%(str(e)))
rospy.service.ServiceException: transport error completing service call: unable to receive data from sender, check sender's logs for details
[mongo_server-1] process has finished cleanly
log file: /home/ori/.ros/log/d72692aa-0a0f-11eb-a2d9-1c697a005041/mongo_server-1*.log
[message_store-3] process has died [pid 7629, exit code 1, cmd /opt/ros/melodic/lib/mongodb_store/message_store_node.py __name:=message_store __log:=/home/ori/.ros/log/d72692aa-0a0f-11eb-a2d9-1c697a005041/message_store-3.log].
log file: /home/ori/.ros/log/d72692aa-0a0f-11eb-a2d9-1c697a005041/message_store-3*.log