search

ironsheep / RPi-Reporter-MQTT2HA-Daemon

Linux service to collect and transfer Raspberry Pi data via MQTT to Home Assistant (for RPi Monitoring)
GNU General Public License v3.0
441 stars 62 forks source link

http://kz0q.com/daemon-releases unavailable, RPi Reporter crashes #114

Open diplix opened 8 months ago

diplix commented 8 months ago

during the night my RPi Reporter installs stopped working. the log indicated that they crashed when trying to check the url http://kz0q.com/daemon-releases — which does not respond (an more?). commenting out calls to getDaemonReleases() mitigated the problem, but i think this should not hapen at all if the update check location happens to be down. also i would suggest to add an option to disable ipdate checks:

# This script checks for pending os updates at a fixed internal in hours [2-24], [Default: 4]
#check_updates_in_hours = 4

why not enable an interval of 0 to disable update checking?

Checklist:

Release with the issue: latest

bsimmo commented 8 months ago

That config setting is for OS updates (so a check on apt/ apt-get, it doesn't alter the script check. The script check is looking to see if a new version of itself is available. I've not looked at the code, it should be disable, it should have some try/else around it.

bsimmo commented 8 months ago

Really it should probably be checking https://github.com/ironsheep/RPi-Reporter-MQTT2HA-Daemon/releases/tag/ and looking for a newer version number. Not sure if GitHub has an easy function for that. v1.8.5

http://kz0q.com isn't secure either and will be @ironsheep website, gone down for some reason. I assume he manual updates that list.

Feel free to amend everything and make it better.

On Sun, 15 Oct 2023, 05:47 felix schwenzel, @.***> wrote:

during the night my RPi Reporter installs stopped working. the log indicated that they crashed when trying to check the url http://kz0q.com/daemon-releases — which does not respond (an more?). commenting out calls to getDaemonReleases() mitigated the problem, but i think this should not hapen at all if the update check location happens to be down. also i would suggest to add an option to disable ipdate checks:

This script checks for pending os updates at a fixed internal in hours [2-24], [Default: 4]

check_updates_in_hours = 4

why not enable an interval of 0 to disable update checking?

Checklist:

  • [x ] I updated to the latest version available
  • [ x] I checked that my MQTT broker is otherwise working

Release with the issue: latest

— Reply to this email directly, view it on GitHub https://github.com/ironsheep/RPi-Reporter-MQTT2HA-Daemon/issues/114, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACYAXN7EV7NYKQ25LGVLW3LX7NTEJANCNFSM6AAAAAA6AXQNQ4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

ironsheep commented 8 months ago

@diplix @bsimmo My server went down, hence the failure. It's been rebooted, and the filesystem space freed up... (attackers dropping large files) sigh...

Thanks for the heads up. The file access should be working once again.

aebgit commented 8 months ago

I do not feel comfortable with this function at all, especially when it makes call to a server where "attackers dropping large files". Can it please become at least optional?

bsimmo commented 8 months ago

This could could be used to get latest version (just tested)

import requests

#https://github.com/ironsheep/RPi-Reporter-MQTT2HA-Daemon.git
owner = "ironsheep"
repo = "RPi-Reporter-MQTT2HA-Daemon"

response = requests.get(f"https://api.github.com/repos/{owner}/{repo}/releases/latest")
latest_version = response.json()["name"]

print(f"The latest version of is {latest_version}")
ironsheep commented 8 months ago

Well, I didn't mean to make light of this outage. I specifically chose a high-availability server for this purpose, which I run and pay for. The server is fully backed up, runs the latest security patches, and is updated every few days with newer security updates. It has had two short outages in the past 4 or 5 years. When something happens, I work with the service host to ensure we are preventing similar failures in the future.

So, while I hear your concerns, I hope to assure you that I am working to make sure this does not cause further impact.

@bsimmo thanks for your suggestion. (It seems I'm always thanking you. I really enjoy your support and activity.) I tried something like that initially, and I believe our current approach has a little bit less runtime impact.

dmshimself commented 6 months ago

I've just had this same problem with the server not being available. Is there an option yet to turn off this checking?

ironsheep commented 6 months ago

@dmshimself i checked the server, no issues. Were you by chance also experiencing any dns issues?

dmshimself commented 6 months ago

No dns issues and the service worked for an hour or so before failing. I guess my general comment is that I’d prefer not to have these hard coded checks in or but to allow as an optional item or if they do fail, to allow the app to continue

On Thu, 14 Dec 2023 at 14:02, Stephen Moraco @.***> wrote:

@dmshimself https://github.com/dmshimself i checked the server, no issues. Were you by chance also experiencing any dns issues?

— Reply to this email directly, view it on GitHub https://github.com/ironsheep/RPi-Reporter-MQTT2HA-Daemon/issues/114#issuecomment-1854932749, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABUU52JFB3IR4RUKP5UQ7A3YJJF2NAVCNFSM6AAAAAA6AXQNQ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJUHEZTENZUHE . You are receiving this because you were mentioned.Message ID: @.***>

aebgit commented 6 months ago

I agree. This should work without such an external dependency.

ironsheep commented 6 months ago

@dmshimself Oh, great suggestion. I'll get a non-impacting fail in my next update.

anderl78 commented 6 months ago

Hello! I like this tool really very much. But the connection to http://kz0q.com for update-check seems not optimal, sorry. And the Problem still exist too, the deamon stopped working (stopps with the known error in log)

I commented out the code (getDaemonReleases() # and load them!) in ISP-RPi-mqtt-daemon.py and it works as it should.

For me the update-check ist not necessary anyway, i can look myself ;-). But if it should still be there for user-service-purposes:

  1. please make a setting for yes/no in config-file
  2. use the github way als written by @bsimmo, way better than the hardcoded site you use atm (at least its the github site)
  3. what about using the update mechanism of HACS?
ironsheep commented 6 months ago

@anderl78 yes, I'm working on repairing this mechanism. Update coming soon... maybe before Christmas? We'll see.

anderl78 commented 6 months ago

@anderl78 yes, I'm working on repairing this mechanism. Update coming soon... maybe before Christmas? We'll see.

Hello! Thank you very much!

matthewhadley commented 5 months ago

EDIT: looks like I was having an unrelated issue resolving dns, which was triggering the same kind of error in this thread.

Is the expectation that this should currently be working but the method is going to be updated to be configurable/fail better? I currently have an issue starting the service with what looks like an error resolving the name. This used to work, but started failing several weeks ago (I'm unsure of the exact date).

admin@raspberrypi:~ $ python3 /opt/RPi-Reporter-MQTT2HA-Daemon/ISP-RPi-mqtt-daemon.py
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 73, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/lib/python3.9/socket.py", line 953, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 394, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 234, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/lib/python3.9/http/client.py", line 1255, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1301, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1250, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1010, in _send_output
    self.send(msg)
  File "/usr/lib/python3.9/http/client.py", line 950, in send
    self.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 200, in connect
    conn = self._new_conn()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0xf611c520>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='kz0q.com', port=80): Max retries exceeded with url: /daemon-releases (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xf611c520>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/RPi-Reporter-MQTT2HA-Daemon/ISP-RPi-mqtt-daemon.py", line 321, in <module>
    getDaemonReleases() # and load them!
  File "/opt/RPi-Reporter-MQTT2HA-Daemon/ISP-RPi-mqtt-daemon.py", line 290, in getDaemonReleases
    response = requests.request('GET', 'http://kz0q.com/daemon-releases', verify=False)
  File "/usr/local/lib/python3.9/dist-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.9/dist-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/requests/adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='kz0q.com', port=80): Max retries exceeded with url: /daemon-releases (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xf611c520>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
bsimmo commented 5 months ago

Just go into the python file and comment out the line as shown above.

It's not been fixed yet. No doubt lack of time and trying to update it all to work nicely in venv for Bookworm type installations.