Core HA Terminal curl command does not work

SNCWilkinson commented 2 months ago

The problem

The command curl -v https://api.octopus.energy times out in the terminal but works outside of the HA platform on other machines. This command has been working until the most recent OS / platform updates.

Same command has worked correctly until this week for over 12 months.

** Previously reported in context of an integration, but as this is a command being run in the terminal

What version of Home Assistant Core has the issue?

core-2024.4.3

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

No response

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

Exact same command works outside of HA platform on multiple machines with different OS's.

mib1185 commented 2 months ago

as mentioned in https://github.com/home-assistant/core/issues/115748#issuecomment-2061129216 this seems to be an issue on your local network configuration, but no HA itself

SNCWilkinson commented 2 months ago

If curl failed for every site I would agree, but it does not, it only fails for one, but it works outside of HA.

Working on a laptop on the same network: curl -v https://api.octopus.energy

Trying 52.209.237.63:443...
Connected to api.octopus.energy (52.209.237.63) port 443
ALPN: curl offers h2,http/1.1
(304) (OUT), TLS handshake, Client hello (1):
CAfile: /etc/ssl/cert.pem
CApath: none
(304) (IN), TLS handshake, Server hello (2):
(304) (IN), TLS handshake, Unknown (8):
(304) (IN), TLS handshake, Certificate (11):
(304) (IN), TLS handshake, CERT verify (15):
(304) (IN), TLS handshake, Finished (20):
(304) (OUT), TLS handshake, Finished (20):
SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256
ALPN: server accepted h2
Server certificate:
subject: CN=octopus.energy
start date: Sep 10 00:00:00 2023 GMT
expire date: Oct 7 23:59:59 2024 GMT
subjectAltName: host "api.octopus.energy" matched cert's "*.octopus.energy"
issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M01
SSL certificate verify ok.
using HTTP/2
[HTTP/2] [1] OPENED stream for https://api.octopus.energy/
[HTTP/2] [1] [:method: GET]
[HTTP/2] [1] [:scheme: https]
[HTTP/2] [1] [:authority: api.octopus.energy]
[HTTP/2] [1] [:path: /]
[HTTP/2] [1] [user-agent: curl/8.4.0]
[HTTP/2] [1] [accept: /]

GET / HTTP/2 Host: api.octopus.energy User-Agent: curl/8.4.0 Accept: /

< HTTP/2 404 < date: Wed, 17 Apr 2024 13:03:49 GMT < content-type: application/json < content-length: 24 < set-cookie: AWSALB=GWRJBuBxBSfCyVXU5ufkP286H5QkMYwiM4ogPkBEyT5OV6GMmOphPIOSftHXo0QNQ75N95kXciHig/mIltUwBf1nvTplnTUDMxgcxytAxKaiIPpxrTsFTxmq+sCH; Expires=Wed, 24 Apr 2024 13:03:49 GMT; Path=/ < set-cookie: AWSALBCORS=GWRJBuBxBSfCyVXU5ufkP286H5QkMYwiM4ogPkBEyT5OV6GMmOphPIOSftHXo0QNQ75N95kXciHig/mIltUwBf1nvTplnTUDMxgcxytAxKaiIPpxrTsFTxmq+sCH; Expires=Wed, 24 Apr 2024 13:03:49 GMT; Path=/; SameSite=None; Secure < server: nginx < x-kraken-correlation-id: 1-661fc8b5-4caaefdf61c0135043a78acd < strict-transport-security: max-age=31536000 < x-content-type-options: nosniff < referrer-policy: same-origin < cross-origin-opener-policy: same-origin < x-frame-options: SAMEORIGIN < vary: Accept-Language, Cookie, origin < content-language: en-gb <
Connection #0 to host api.octopus.energy left intact {"detail": "Not found."}%

curl in HA with a different public API call:

mib1185 commented 2 months ago

If curl failed for every site I would agree, but it does not, it only fails for one, but it works outside of HA.

it's exactly the opposite - if it would fail for every URL it would be an issue in HA (or HA OS), but when only one URL is affected, it is most properly an issue on your network (eq. a firewall)

mib1185 commented 2 months ago

try traceroute -p 443 api.octopus.energy in the Terminal, this might help to find the blocking device

SNCWilkinson commented 2 months ago

Nothing on local network causing issues, gets out into the internet without being blocked.

It is ONLY the HA platform that is not working and it has been fine for over 12 months until this week, with no changes on any platform other than the updates to HA OS / Core.

mib1185 commented 2 months ago

what is wget https://api.octopus.energy and nc -v -w 1 api.octopus.energy 443 showing?

the first 2 hops https://github.com/home-assistant/core/issues/115760#issuecomment-2061273682 in should be local (private) IPs, 1st is the docker engine running on your HA OS, 2nd should be your internet router - is this correct?

i've checked twice on my HA and it can reach https://api.octopus.energy via curl (as i was already expecting)

SNCWilkinson commented 2 months ago

docker
router
IP

wget times out

nc -v -w 1 api.octopus.energy 443 nc: api.octopus.energy (52.215.247.153:443): Operation timed out

I was 99% sure this was likely something local with my setup, but as it was working until recently that's what's thrown me. I can't see any obvious errors with any aspect, all other integrations work fine (internet and local) just this one :/

mib1185 commented 2 months ago

as it works on my end and only one single URL is affected, i'm still on the point, that it is not a general issue in HA.

you can try to downgrade (don't forget to take a backup first)

how do you exactly run HA-OS?

SNCWilkinson commented 2 months ago

Windows Server running Hyper-V, then running HA OS / docker within an Hyper-V image.

Been absolutely 100% trouble free until this week. Downgrades it is to see if that fixes things :(

mib1185 commented 2 months ago

@SNCWilkinson based on https://github.com/home-assistant/core/issues/115748#issuecomment-2064949874 i think the issue is solved?

issue-triage-workflows[bot] commented 2 months ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

home-assistant / core