Tendrl / node-agent

A python agent local to every managed storage node in the sds cluster
GNU Lesser General Public License v2.1
4 stars 14 forks source link

Somtimes node-agent message socket file "message.sock" is missing #850

Closed GowthamShanmugam closed 5 years ago

GowthamShanmugam commented 5 years ago

When node-agent service is started socket file "message.sock" was created under /var/run/tendrl. Using socket file only node-agent message socket server listen and accept the connections. Sometimes when we restart node-agent socket file is not created and node-agent message handler thread is failing.

I have seen this scenario a few times after installation done from tendrl-ansible in a fresh machine.

All other components in a node are kept raising the following exception:

Sep 16 03:42:02 GlusterWebAdmin tendrl-monitoring-integration: Traceback (most recent call last): Sep 16 03:42:02 GlusterWebAdmin tendrl-monitoring-integration: File "/usr/lib/python2.7/site-packages/tendrl/commons/event.py", line 38, in _write Sep 16 03:42:02 GlusterWebAdmin tendrl-monitoring-integration: self.sock.connect(self.socket_path) Sep 16 03:42:02 GlusterWebAdmin tendrl-monitoring-integration: File "/usr/lib64/python2.7/socket.py", line 224, in meth Sep 16 03:42:02 GlusterWebAdmin tendrl-monitoring-integration: return getattr(self._sock,name)(*args) Sep 16 03:42:02 GlusterWebAdmin tendrl-monitoring-integration: error: [Errno 2] No such file or directory Sep 16 03:42:02 GlusterWebAdmin tendrl-monitoring-integration: Unable to pass the message into socket.{"integration_id": null, "publisher": "monitoring_integration", "job_id": null, "timestamp": "2018-09-16T03:42:02.051258+00:00", "caller": {"function": "load_definition", "line_no": 50, "filename": "/usr/lib/python2.7/site-packages/tendrl/commons/objects/init.py"}, "payload": {"message": "Load definitions (.yml) for namespace.tendrl.objects.Job"}, "priority": "debug", "parent_id": null, "node_id": "a6ed6301-260d-4a4d-8a35-62f980122ee1", "flow_id": null, "message_id": "a178660d-7333-43d0-a7ef-aace791fbf30"}

GowthamShanmugam commented 5 years ago

this issue is fixed