[Bug] Ping not showing response when API gets data. How to troubleshoot deeper

[corvock]

Description

I have set up Radarr as a service. The widget works as expected. When i added ping: to the definition, it always returns 401 in the box where other services return a time in ms. If that is the http code there is no auth on it that would block internal requests.

curl -L -k returns html content.

I am unsure what the next steps to troubleshoot this should be.

Steps to reproduce

open homepage with radarr configured to ping, i get a number in red which is unexpected when the machine is accessible by api for the widget

homepage version

0.5.2

Installation method

Source

Configuration

- Radarr:
        icon: radarr.png
        href: https://url
        ping: http://IP:7878
        description: Movies
        widget:
          type: radarr
          url: http://IP:7878
          key: <redacted>

Container Logs

No response

Browser Logs

No response

Other

My proxmox hosts do the same thing with different numbers in red. They're not on the same subnet, so added issues. Looking to find out how to replicate what "ping" does on the command line to troubleshoot them or get an error that can show more info.

[shamoon]

ping runs from inside the container the same as API fetching, so yes if API is reaching it then ping can reach the host. But, the API endpoint may not respond to requests the same way as an API call. Ping merely sends a head request to that URL, which would be the curl equivalent of curl --head http://IP:7878 (from inside the container). In this case, radarr is throwing a 401 unauthorized. Depending on your setup, you may want to use your href instead, but again, ping isnt going to do anything fancy like get past a reverse-proxy, login etc. its just checking a URL returns a valid http status code. Its intentionally kinda basic.

Let me know if you need more explanation

[corvock]

Thanks! that command is letting me see why it is failing. which is helpful. Still baffled why when Radarr has all auth disabled it is still saying unauthorized, but that is another rabbit hole.

Last question. Since Proxmox is replying "HTTP/1.1 501 method 'HEAD' not available" .. that is just not going to work unless they make changes to their web server software?

Thanks again!

--Ryan

[shamoon]

Yes. We debated a little bit whether to use HEAD or not but its so much faster. That proxmox URL youre talking about responds with e.g. a 200 to a GET request?

[corvock]

yup

[shamoon]

Interesting. Do you have any other services like this that 501 the HEAD request but 200 a GET?

[corvock]

nope. Just the proxmox stuff. Their backup server throws a different error .. 400

[JazzFisch]

From a security perspective, it makes sense. Don't allow verbs you don't support to reduce attack surface. As a fellow Proxmox user, I too was frustrated I couldn't turn on ping for my Proxmox instance.

[shamoon]

Actually I dont know why they would ever do that, I think HEAD is basically a shorter GET, basically all it does is save bandwidth. If the HEAD request is rejected presumably an attacker would just try a GET?

Anyway, a simple idea would be to fall back to GET if HEAD fails. The only disadvantage of this is that it would add an unnecessary extra attempt for true failures, of course. Since I dont know how many services do this (disable HEAD) I dont know if this fix would be 'worth it'. Then again I dont expect it would be a significant time increase.

I made a branch for this, @JazzFisch (since now I know you have a proxmox instance to test) or anyone else if you want to give that a try: https://github.com/benphelps/homepage/tree/ping-fallback

[corvock]

I can install in the morning and see how it does

[shamoon]

That change does seem to mitigate the Proxmox issue (confirmed by another contrib) but I think we're going to hold off on it. Seems like the whole rejecting HEAD requests is hopefully pretty rare. If it turns out to be an issue for lots of services we can of course re-address in the future.

[brhahlen]

Not sure if it's related, but I have a bunch of services that give a 405 back on the ping. If it's not related, let me know, I'll create a separate issue.

[corvock]

possible to make the failover opt in? so it retains the expected behavior unless a setting exists?

[shamoon]

Not sure if it's related, but I have a bunch of services that give a 405 back on the ping.

Exactly which services are giving you a 405?

[brhahlen]

Not sure if it's related, but I have a bunch of services that give a 405 back on the ping.

Exactly which services are giving you a 405?

Traefik, HomeAssistant, Code Server, Transmission, ESPHome.

[shamoon]

Transmission gives a 401, which is different, thats no auth.

Code Server works fine for me:

Im not going to test them all but Im interested in which reject HEAD but allow GET as that is the only potential change on the table here (a fallback to GET).

Ping is meant to be simple e.g. to check the response time of a publicly-facing website. Btw, if you have a widget and its working, ping is redundant unless your goal is to check its public-facing endpoint. But again, if you have auth, reverse-proxy etc. its not going to negotiate any of those. Our goal is to keep it simple even if it doesnt apply in 100% of cases.

[corvock]

So my issues could be use case. I have API's hitting the reverse proxy, and ping hitting the internal ip. So with Sonarr the API says that the external flow in is working and pink tells me the machine itself is working. So that gives me two bits of information in one widget. With proxmox, i have multiple clustered hosts, so the pings tell me if those hosts are up. Obv when there are larger problems I am knowing before i look at homepage, but it is a nice warm fuzzy that they're all green. I have a machine that keeps homepage in kiosk mode sitting on my rack because i like the quick glance of what is going on. Externally I hit it more for bookmarks/service links but it is still nice to see.

I have found that Authelia responds with a 405 and n8n is responding with 401 to the head requests and both respond 200 to get requests

On Tue, Nov 29, 2022 at 1:53 PM shamoon @.***> wrote:

Transmission gives a 401, which is different, thats no auth.

Code Server works fine for me: [image: Screen Shot 2022-11-29 at 1 49 09 PM] https://user-images.githubusercontent.com/4887959/204655373-bfaca4f3-a87c-40b1-a57f-c1d5e7a1aeb0.png

Im not going to test them all but Im interested in which reject HEAD but allow GET as that is the only potential change on the table here (a fallback to GET).

Ping is meant to be simple e.g. to check the response time of a publicly-facing website. Btw, if you have a widget and its working, ping is redundant unless your goal is to check its public-facing endpoint. But again, if you have auth, reverse-proxy etc. its not going to negotiate any of those. Our goal is to keep it simple even if it doesnt apply in 100% of cases.

— Reply to this email directly, view it on GitHub https://github.com/benphelps/homepage/issues/572#issuecomment-1331370081, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACPBVULCNGX6DQMI72MTVFLWKZ3PNANCNFSM6AAAAAASN4WNZM . You are receiving this because you authored the thread.Message ID: @.***>

[JigSawFr]

Also facing the same problem on my side with multiples services including sonarr, radarr Using Authelia as OP too.

[shamoon]

This was all discussed when we considered implementing this feature at all. If a service is behind authelia, then ping (regardless of whether it uses HEAD or GET) is pointless because all you’re pinging is the authelia login page. Ping is not going to login past your reverse proxy, etc. So either don’t use ping in that situation or if you do want to use it, you’d have to ping some presumably internal endpoint that’s open.

If people wanna get fancy and implement something on their reverse proxy that allows pings (again just a HEAD request) e.g. from a their own host or something like that, then sure. Otherwise, this is working as intended.

I can make the docs clearer about this.

[corvock]

I'm not trying to ping through the reverse proxy. I am hitting the services directly. Homepage is on the same side of the proxy as what i am trying to ping.

On Tue, Nov 29, 2022 at 4:09 PM shamoon @.***> wrote:

This was all discussed when we considered implementing this feature at all. If a service is behind authelia, then ping (regardless of whether it uses HEAD or GET) is pointless because all you’re pinging is the authelia login page. Ping is not going to login past your reverse proxy, etc. So either don’t use ping in that situation or if you do want to use it, you’d have to ping some presumably internal endpoint that’s open.

If people wanna get fancy and implement something on their reverse proxy that allows pings (again just a HEAD request) e.g. from a their own host or something like that, then sure. Otherwise, this is working as intended.

I can make the docs clearer about this.

— Reply to this email directly, view it on GitHub https://github.com/benphelps/homepage/issues/572#issuecomment-1331479822, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACPBVULNUGBRJCOV7FAP4W3WK2LL7ANCNFSM6AAAAAASN4WNZM . You are receiving this because you authored the thread.Message ID: @.***>

[JigSawFr]

@shamoon yes, sorry I was not clear in my message, like @corvock just said, i'm pinging services directly, so in this case authelia/traefik is not involved for me as they are only used for external purposes, but for communication and pings, i'm using directly services. It's working for many services but in my case i'm facing 401 with sonarr v4/radarr v4, for ping feature only. Was thinking it was related to a feature of sonarr, but no effect.

[corvock]

@JigSawFr that is interesting. My Sonarr is working fine for ping, but my Radarr is giving a 401. Both are v4 and at current vers

[JigSawFr]

@JigSawFr that is interesting. My Sonarr is working fine for ping, but my Radarr is giving a 401. Both are v4 and at current vers

Can you tell me which docker images are you using for the two apps ? (and their tag)

[brhahlen]

Transmission gives a 401, which is different, thats no auth.

Code Server works fine for me:

Im not going to test them all but Im interested in which reject HEAD but allow GET as that is the only potential change on the table here (a fallback to GET).

Ping is meant to be simple e.g. to check the response time of a publicly-facing website. Btw, if you have a widget and its working, ping is redundant unless your goal is to check its public-facing endpoint. But again, if you have auth, reverse-proxy etc. its not going to negotiate any of those. Our goal is to keep it simple even if it doesnt apply in 100% of cases.

That makes sense. Transmission for me has no auth, but gives the 405. Radarr gives me the 401 though.

I am doing the ping to the Reverse Proxy (Traefik) URL, which might be causing the issue. The things that is interesting to me then, is why some URLs work, but others don't.

I did try the curl commands from my local machine, and this is what I got (both for code):

$ curl --head https://code.domain.tld
HTTP/2 405
access-control-allow-origin: *
content-type: text/plain
date: Wed, 30 Nov 2022 08:02:01 GMT
strict-transport-security: max-age=6307200; includeSubDomains; preload
vary: Origin

$ curl -i https://code.domain.tld
HTTP/2 200
access-control-allow-origin: *
content-security-policy: default-src 'self'; img-src 'self' https: data: blob:; media-src 'self'; script-src 'self' 'unsafe-eval' 'sha256-stuffhere=' 'sha256-stuffhere=' 'sha256-stuffhere=' 'sha256-stuffhere=' http://code.domain.tld; child-src 'self'; frame-src 'self' https://*.vscode-cdn.net data:; worker-src 'self' data:; style-src 'self' 'unsafe-inline'; connect-src 'self' ws: wss: https:; font-src 'self' blob:; manifest-src 'self';
content-type: text/html
date: Wed, 30 Nov 2022 08:02:25 GMT
strict-transport-security: max-age=6307200; includeSubDomains; preload
vary: Origin

<!-- Copyright (C) Microsoft Corporation. All rights reserved. -->
<!DOCTYPE html>
<html>
....
</html>

So I guess the fallback ping will work for this as well?

[shamoon]

Ok, after all the back-and-forth it seems like the cost / benefit is in favor of the fallback option. It will not change some of the issues raised here but in the end it seems common-enough that some services reject a HEAD in favor of GET so its reasonable to include. This can be tested out in main for the curious.

[brhahlen]

@shamoon Just pulled main and this indeed fixed the issue. Everything is responding now to the ping!

Thanks!

[corvock]

Same here. Thanks!

[henryxrl]

I'm also wondering about 401 unauthorized error. If we won't allow 401 as a pass, is it possible to have the option of filling in a username/password just like what Uptime Kuma does?

[JazzFisch]

I'm also wondering about 401 unauthorized error. If we won't allow 401 as a pass, is it possible to have the option of filling in a username/password just like what Uptime Kuma does?

Create a new feature request with this detail.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new discussion for related concerns.