MaxIV-KitsControls / dev-maxiv-raspberry_pi

Tango DS for the Raspberry Pi GPIO
GNU General Public License v3.0
3 stars 4 forks source link

Tango device RaspberryPiIO getting into FAULT state #8

Open reszelaz opened 5 years ago

reszelaz commented 5 years ago

Hi,

We are using commit: 95a32ee (merge pull request #6 from MaxIV-KitsControls/issue-4).

We have observed this issue twice already. The Tango device RaspberryPiIO gets into the FAULT state.

----------------------------------------------------
Command: ald/io/raspberrypiio-01/State
Duration: 2 msec
Output argument(s) :
FAULT
----------------------------------------------------
Command: ald/io/raspberrypiio-01/Status
Duration: 0 msec
Output argument(s) :
The device is in FAULT state.

while the tcpserver on the raspberry pi is running:

pi@raspberrypi:~ $ ps -ef \| grep tcp
--
root       451   419  0 Aug14 tty7     00:51:43 /usr/lib/xorg/Xorg :0 -seat seat0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7 -novtswitch
tango      838     1  0 Aug14 ?        00:00:00 /usr/lib/tango/DataBaseds 2 -ORBendPoint giop:tcp::10000
pi        1280  1260  0 16:20 pts/0    00:00:00 grep --color=auto tcp
pi       20939     1  0 Oct17 ?        00:38:09 /usr/bin/python .local/bin/tcpserver-raspberry_pi

Restarting of the RaspberryPiIO DS from Starter did not work. I had to restart the tcpserver-raspberry_pi on the raspberry pi and again restart the RaspberryPiIO DS.

This time it took like 35 days since we restarted it last time, but we don't know at which time it went into the FAULT state.

First of all, have you observed this kind of behavior?

In order to improve the post mortem analysis I uncommented the debug traces from catch_connection_error decorator and enabled the Tango logging. This way, at least, we should know at which moment it happens (there is polling enabled on some of the attributes). Do you know any other way on how to investigate this issue?

Many thanks in advance!

jensun84 commented 5 years ago

Hi Zibi, Interesting. If the TCP/IP server is indeed running on the Raspberry, it should be enough with an init on the Tango DS. We have not experienced this before. Let us know if it happens again, Thanks a lot, Jens