search

robotics-in-concert / rocon_multimaster

Key components for ros multimaster systems
11 stars 19 forks source link

Gateway redis connection error on lost hub #251

Closed stonier closed 10 years ago

stonier commented 10 years ago

This is occuring in get_remote_connection_state when it tries to retrieve the advertisements after a hub has gone down, but before the gateway has processed the disconnection.

[INFO] [WallTime: 1393978936.804867] Gateway : registering on the hub [Concert Tutorial]
[INFO] [WallTime: 1393978960.715993] Gateway : Lost connection with hub. Attempting to disconnect...
[ERROR] [WallTime: 1393978960.720708] Gateway: unable to update latency stats for guimul5573cd64ff8347b4867658cc2b113203
[INFO] [WallTime: 1393978960.725639] Gateway : lost connection to the hub [Concert Tutorial][192.168.10.129:6380]
Traceback (most recent call last):
  File "/mnt/zaphod/code/ros/rocon/src/rocon_multimaster/rocon_gateway/scripts/gateway.py", line 22, in <module>
    gateway.spin()
  File "/mnt/zaphod/code/ros/rocon/src/rocon_multimaster/rocon_gateway/src/rocon_gateway/gateway_node.py", line 69, in spin
    self._gateway.spin()
  File "/mnt/zaphod/code/ros/rocon/src/rocon_multimaster/rocon_gateway/src/rocon_gateway/gateway.py", line 75, in spin
    self.watcher_thread.start()
  File "/mnt/zaphod/code/ros/rocon/src/rocon_multimaster/rocon_gateway/src/rocon_gateway/watcher_thread.py", line 76, in start
    self._gateway.update_pulled_interface(connections, remote_gateway_hub_index)
  File "/mnt/zaphod/code/ros/rocon/src/rocon_multimaster/rocon_gateway/src/rocon_gateway/gateway.py", line 176, in update_pulled_interface
    remote_connections[remote_gateway].update(hub.get_remote_connection_state(remote_gateway))
  File "/mnt/zaphod/code/ros/rocon/src/rocon_multimaster/rocon_gateway/src/rocon_gateway/gateway_hub.py", line 460, in get_remote_connection_state
    public_interface = self._redis_server.smembers(key)
  File "/mnt/zaphod/code/ros/rocon/src/rocon_tools/rocon_python_redis/src/rocon_python_redis/client.py", line 919, in smembers
    return self.execute_command('SMEMBERS', name)
  File "/mnt/zaphod/code/ros/rocon/src/rocon_tools/rocon_python_redis/src/rocon_python_redis/client.py", line 349, in execute_command
    connection.send_command(*args)
  File "/mnt/zaphod/code/ros/rocon/src/rocon_tools/rocon_python_redis/src/rocon_python_redis/connection.py", line 300, in send_command
    self.send_packed_command(self.pack_command(*args))
  File "/mnt/zaphod/code/ros/rocon/src/rocon_tools/rocon_python_redis/src/rocon_python_redis/connection.py", line 282, in send_packed_command
    self.connect()
  File "/mnt/zaphod/code/ros/rocon/src/rocon_tools/rocon_python_redis/src/rocon_python_redis/connection.py", line 229, in connect
    raise ConnectionError(self._error_message(e))
rocon_python_redis.exceptions.ConnectionError: Error 111 connecting 192.168.10.129:6380. Connection refused.
stonier commented 10 years ago

Another one:

[INFO] [WallTime: 1393980498.441839] Gateway : Lost connection with hub. Attempting to disconnect...
Traceback (most recent call last):
  File "/mnt/zaphod/code/ros/rocon/src/rocon_multimaster/rocon_gateway/scripts/gateway.py", line 22, in <module>
    gateway.spin()
  File "/mnt/zaphod/code/ros/rocon/src/rocon_multimaster/rocon_gateway/src/rocon_gateway/gateway_node.py", line 69, in spin
    self._gateway.spin()
  File "/mnt/zaphod/code/ros/rocon/src/rocon_multimaster/rocon_gateway/src/rocon_gateway/gateway.py", line 75, in spin
    self.watcher_thread.start()
  File "/mnt/zaphod/code/ros/rocon/src/rocon_multimaster/rocon_gateway/src/rocon_gateway/watcher_thread.py", line 72, in start
    remote_gateway_hub_index = self._hub_manager.create_remote_gateway_hub_index()
  File "/mnt/zaphod/code/ros/rocon/src/rocon_multimaster/rocon_gateway/src/rocon_gateway/hub_manager.py", line 78, in create_remote_gateway_hub_index
    for remote_gateway in hub.list_remote_gateway_names():
  File "/mnt/zaphod/code/ros/rocon/src/rocon_multimaster/rocon_gateway/src/rocon_gateway/gateway_hub.py", line 409, in list_remote_gateway_names
    gateway_keys = self._redis_server.smembers(self._redis_keys['gatewaylist'])
  File "/mnt/zaphod/code/ros/rocon/src/rocon_tools/rocon_python_redis/src/rocon_python_redis/client.py", line 919, in smembers
    return self.execute_command('SMEMBERS', name)
  File "/mnt/zaphod/code/ros/rocon/src/rocon_tools/rocon_python_redis/src/rocon_python_redis/client.py", line 346, in execute_command
    return self.parse_response(connection, command_name, **options)
  File "/mnt/zaphod/code/ros/rocon/src/rocon_tools/rocon_python_redis/src/rocon_python_redis/client.py", line 356, in parse_response
    response = connection.read_response()
  File "/mnt/zaphod/code/ros/rocon/src/rocon_tools/rocon_python_redis/src/rocon_python_redis/connection.py", line 307, in read_response
    self.disconnect()
  File "/mnt/zaphod/code/ros/rocon/src/rocon_tools/rocon_python_redis/src/rocon_python_redis/connection.py", line 274, in disconnect
    self._sock.close()
AttributeError: 'NoneType' object has no attribute 'close'