Closed pepelisu closed 4 years ago
Can you give me more details? branch? frequency?
I am unsing dev branch [dev] d48352. The frequency,... I am still figuring that out, It did not happen again since the last restart from the services. However it was happening after one to three days since the tracking started. I will keep an eye on it and if this does not happen again I will close the issue.
I think this is happening when device_scanner is used and there is no ethoscopes online. It just happened now when all the ethoscopes where power off, so device_scanner returns an empty object.
The error is coming from the line 299 in etho_db.py
, in the line above if function self._edb.getEthoscope('all'.asdict=True)
is returning 0 or -1 due to an error on the commit. When manually checking the sqlite the database appers to be locked. After a deeper inspection there is 33 concurrent connections locking the database and giving a Permission denied error
:
The command fuser /etc/ethoscope-node.db
gives:
Cannot stat file /proc/1229/fd/0: Permission denied
Cannot stat file /proc/1229/fd/1: Permission denied
Cannot stat file /proc/1229/fd/2: Permission denied
Cannot stat file /proc/1229/fd/3: Permission denied
Cannot stat file /proc/1229/fd/4: Permission denied
Cannot stat file /proc/1229/fd/5: Permission denied
Cannot stat file /proc/1229/fd/6: Permission denied
Cannot stat file /proc/1229/fd/7: Permission denied
Cannot stat file /proc/1229/fd/8: Permission denied
Cannot stat file /proc/1229/fd/9: Permission denied
Cannot stat file /proc/1229/fd/10: Permission denied
Cannot stat file /proc/1229/fd/11: Permission denied
Cannot stat file /proc/1229/fd/12: Permission denied
Cannot stat file /proc/1229/fd/13: Permission denied
Cannot stat file /proc/1229/fd/14: Permission denied
Cannot stat file /proc/1229/fd/15: Permission denied
Cannot stat file /proc/1229/fd/16: Permission denied
Cannot stat file /proc/1229/fd/18: Permission denied
Cannot stat file /proc/1229/fd/19: Permission denied
Cannot stat file /proc/1229/fd/20: Permission denied
Cannot stat file /proc/1229/fd/21: Permission denied
Cannot stat file /proc/1229/fd/22: Permission denied
Cannot stat file /proc/1229/fd/23: Permission denied
Cannot stat file /proc/1229/fd/24: Permission denied
Cannot stat file /proc/1229/fd/25: Permission denied
Cannot stat file /proc/1229/fd/26: Permission denied
Cannot stat file /proc/1229/fd/27: Permission denied
Cannot stat file /proc/1229/fd/28: Permission denied
Cannot stat file /proc/1229/fd/29: Permission denied
Cannot stat file /proc/1229/fd/30: Permission denied
Cannot stat file /proc/1229/fd/31: Permission denied
Cannot stat file /proc/1229/fd/32: Permission denied
Cannot stat file /proc/1229/fd/33: Permission denied
Process 1229 is /usr/bin/gnome-shell
, by killing the process, it spawns a new process and the same thing happens again.
Changing the file permissions to 666 does not solve the problem.
Stopping the services ethoscope_node
and ethoscope_backup
and killing the process does not unlock the database, still too many connections?
Restarting the node seems to be the only way to stop the process to re-spawn.
Sorry, I am confused. Is it two different issues we're talking about? The code on etho_db.py is the following:
I don't understand why there are so many closed connections, though. It's not the case here:
[gg@node ~]$ sudo fuser /etc/ethoscope-node.db
/etc/ethoscope-node.db: 53238 103807
I am not sure if the issues are related. What I know is that backup time disappears and the only error that I found logged in the system was that problem with sqlite.
I would suggest to add some error handling in line 299, because when in line 293 self.executeSQL(sql_get_ethoscope)
returns -1 and if asdict == True, then row[0] will fail. Checking if row == -1 and log the error would be necessary.
I could not find the origin of so many connections but I am looking into it.
Ok, I've changed to this
https://github.com/gilestrolab/ethoscope/commit/7eb99d719e40b101ed84786649e432a9176ad090
Now, we would have to understand why you get permission denied (and I never do). Could this be due to some settings in your mariadb? For now, I'll close this but feel free to reopen if you manage to reproduce.
Time since backup in the list of the connected ethoscopes is dissapearing, and showing and empty column.
Ethoscope_backup.service seems to run ok, however ethoscope_node.service shows this error (when checking the status)
Workaround: Restart ethoscope_node.service and ethoscope_backup.service