search

Ysurac / openmptcprouter-vps

OpenMPTCProuter VPS scripts
GNU General Public License v3.0
158 stars 69 forks source link

omr-admin.service stuck in reboot loop #62

Open donbecker opened 2 years ago

donbecker commented 2 years ago

Hi,

I've deployed the VPS using the scripts on AWS EC2, using a Debian 10 image.

After using it for a few months and then rebooting, it appears that omr.admin.service is in a reboot loop:

tail -f /var/log/syslog

Dec 8 19:19:39 ip-172-31-14-148 OMR-Service: Restart OMR-Admin Dec 8 19:19:39 ip-172-31-14-148 systemd[1]: Stopping OMR-Admin... Dec 8 19:19:39 ip-172-31-14-148 systemd[1]: omr-admin.service: Main process exited, code=killed, status=15/TERM Dec 8 19:19:39 ip-172-31-14-148 systemd[1]: omr-admin.service: Succeeded. Dec 8 19:19:39 ip-172-31-14-148 systemd[1]: Stopped OMR-Admin. Dec 8 19:19:39 ip-172-31-14-148 systemd[1]: Started OMR-Admin. Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: Traceback (most recent call last): Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: File "/usr/bin/omr-admin.py", line 773, in Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: omr_config_data = json.load(f) Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: File "/usr/lib/python3.7/json/init.py", line 296, in load Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, kw) Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: File "/usr/lib/python3.7/json/init.py", line 348, in loads Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: return _default_decoder.decode(s) Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: File "/usr/lib/python3.7/json/decoder.py", line 337, in decode Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: obj, end = self.raw_decode(s, idx=_w(s, 0).end()) Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: raise JSONDecodeError("Expecting value", s, err.value) from None Dec 8 19:19:44 ip-172-31-14-148 omr-admin.py[5750]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Dec 8 19:19:45 ip-172-31-14-148 systemd[1]: omr-admin.service: Main process exited, code=exited, status=1/FAILURE Dec 8 19:19:45 ip-172-31-14-148 systemd[1]: omr-admin.service: Failed with result 'exit-code'. Dec 8 19:19:45 ip-172-31-14-148 systemd[1]: omr-admin.service: Service RestartSec=100ms expired, scheduling restart. Dec 8 19:19:45 ip-172-31-14-148 systemd[1]: omr-admin.service: Scheduled restart job, restart counter is at 1. Dec 8 19:19:45 ip-172-31-14-148 systemd[1]: Stopped OMR-Admin. Dec 8 19:19:45 ip-172-31-14-148 systemd[1]: Started OMR-Admin. Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: Traceback (most recent call last): Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: File "/usr/bin/omr-admin.py", line 773, in Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: omr_config_data = json.load(f) Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: File "/usr/lib/python3.7/json/init.py", line 296, in load Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, kw) Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: File "/usr/lib/python3.7/json/init.py", line 348, in loads Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: return _default_decoder.decode(s) Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: File "/usr/lib/python3.7/json/decoder.py", line 337, in decode Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: obj, end = self.raw_decode(s, idx=_w(s, 0).end()) Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: raise JSONDecodeError("Expecting value", s, err.value) from None Dec 8 19:19:51 ip-172-31-14-148 omr-admin.py[5770]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Dec 8 19:19:52 ip-172-31-14-148 OMR-Service: Restart OMR-Admin

It appears that I'm still successfully passing traffic thru the VPS but I'd like to fix this.

Please let me know if you need any other info to help.

Ysurac commented 2 years ago

I would need the VPS script release used (available in /etc/motd). Check also that /etc/openmptcprouter-vps-admin/omr-admin-config.json

donbecker commented 2 years ago

"File "/usr/bin/omr-admin.py", line 773"

lines 772-773:

with open('/etc/openmptcprouter-vps-admin/omr-admin-config.json') as f: omr_config_data = json.load(f)

So I suspect this is throwing an error trying to load the config file.

Checking the omr-admin-config.json file, I find it's completely empty, last edit 11/15. I've never modified this file.

Luckily I had a manual backup of the entire box dated 11/13, and I was able to copy the config over.

With 2 cellular internet connections and omr-admin service in reboot loop I was getting approx 60M down.

Once I updated the config, the service restarted and it has not restarted since.

I now get approx 96M down.

Back in the Luci panel for my local OpenMPTCP router (System -> OpenMPTCPRouter -> Status).... the VPS server in the diagram now shows green (was showing some error about admin connection before).

Any idea what might cause this config file to be completely empty?