search

cloudflare / cloudflared

Cloudflare Tunnel client (formerly Argo Tunnel)
https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/tunnel-guide
Apache License 2.0
8.41k stars 732 forks source link

After working for a year today I got this error Unable to reach the origin service.🐛 #1225

Open Knowix opened 2 months ago

Knowix commented 2 months ago

Describe the bug Hey guys,

I've been using Cloudflare Argo tunnel for more than a year now. I installed it by following ibracrop's video on YouTube, and it worked like a charm. But today for some reason the tunnel stopped working properly. I have no experience in this field, I only followed a YouTube video. And also I didn't push any updates or changes to my network.

I'd be grateful if someone could help me solve my problem.

This is the error message that I see:

2024-04-02T22:50:44Z INF Starting tunnel tunnelID=522e3fca-111c-44fc-9379-e143d5254a46 2024-04-02T22:50:44Z INF Version 2024.1.4 2024-04-02T22:50:44Z INF GOOS: linux, GOVersion: go1.21.5-devel-cf, GoArch: amd64 2024-04-02T22:50:44Z INF Settings: map[cred-file:/home/nonroot/.cloudflared/522e3fca-111c-44fc-9379-e143d5254a46.json credentials-file:/home/nonroot/.cloudflared/522e3fca-111c-44fc-9379-e143d5254a46.json no-autoupdate:true] 2024-04-02T22:50:44Z INF Generated Connector ID: 37222e91-dd6c-465d-942e-df7057975c6d 2024-04-02T22:50:54Z ERR Failed to fetch features, default to disable error="lookup cfd-features.argotunnel.com on 127.0.0.11:53: read udp 127.0.0.1:41168->127.0.0.11:53: i/o timeout" 2024-04-02T22:50:54Z ERR update check failed error="Get \"https://update.argotunnel.com?arch=amd64&clientVersion=2024.1.4&os=linux\": dial tcp: lookup update.argotunnel.com on 127.0.0.11:53: read udp 127.0.0.1:60079->127.0.0.11:53: i/o timeout" 2024-04-02T22:51:04Z WRN Unable to lookup protocol percentage. 2024-04-02T22:51:04Z INF Initial protocol http2 2024-04-02T22:51:04Z INF ICMP proxy will use 172.18.0.3 as source for IPv4 2024-04-02T22:51:04Z INF ICMP proxy will use :: as source for IPv6 2024-04-02T22:51:24Z INF Starting metrics server on 127.0.0.1:44535/metrics 2024-04-02T22:51:44Z INF Tunnel server stopped

I also tried to start the tunnel from the console where I could start it and in Cloudflare it became Healthy but then it went back to Down in a few mins.

And I see these logs in the console.

2024-04-02T22:27:19Z ERR error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: remote error: tls: unrecognized name" connIndex=1 event=1 ingressRule=0 originService=https://192.168.2.81:18443 2024-04-02T22:27:19Z ERR Request failed error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: remote error: tls: unrecognized name" connIndex=1 dest=https://download.knowix.pro/ event=0 ip=198.41.192.67 type=http 2024-04-02T22:27:19Z ERR error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: remote error: tls: unrecognized name" connIndex=1 event=1 ingressRule=0 originService=https://192.168.2.81:18443 2024-04-02T22:27:19Z ERR Request failed error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: remote error: tls: unrecognized name" connIndex=1 dest=https://download.knowix.pro/favicon.ico event=0 ip=198.41.192.67 type=http ^C2024-04-02T22:27:46Z INF Initiating graceful shutdown due to signal interrupt ... 2024-04-02T22:27:47Z INF Unregistered tunnel connection connIndex=3 event=0 ip=198.41.192.47 2024-04-02T22:27:47Z ERR Failed to serve quic connection error="Application error 0x0 (remote)" connIndex=3 event=0 ip=198.41.192.47 2024-04-02T22:27:47Z ERR Serve tunnel error error="Application error 0x0 (remote)" connIndex=3 event=0 ip=198.41.192.47 2024-04-02T22:27:47Z INF Retrying connection in up to 1s connIndex=3 event=0 ip=198.41.192.47 2024-04-02T22:27:47Z INF Unregistered tunnel connection connIndex=0 event=0 ip=198.41.200.193 2024-04-02T22:27:47Z ERR Failed to serve quic connection error="Application error 0x0 (remote)" connIndex=0 event=0 ip=198.41.200.193 2024-04-02T22:27:47Z ERR Serve tunnel error error="Application error 0x0 (remote)" connIndex=0 event=0 ip=198.41.200.193 2024-04-02T22:27:47Z INF Retrying connection in up to 1s connIndex=0 event=0 ip=198.41.200.193 2024-04-02T22:27:47Z INF Unregistered tunnel connection connIndex=1 event=0 ip=198.41.192.67 2024-04-02T22:27:47Z ERR writing finish: Application error 0x0 (remote) 2024-04-02T22:27:47Z ERR Failed to serve quic connection error="Application error 0x0 (remote)" connIndex=1 event=0 ip=198.41.192.67 2024-04-02T22:27:47Z ERR Serve tunnel error error="Application error 0x0 (remote)" connIndex=1 event=0 ip=198.41.192.67 2024-04-02T22:27:47Z INF Retrying connection in up to 1s connIndex=1 event=0 ip=198.41.192.67 2024-04-02T22:27:47Z INF Unregistered tunnel connection connIndex=2 event=0 ip=198.41.200.73 2024-04-02T22:27:47Z ERR Failed to serve quic connection error="context canceled" connIndex=2 event=0 ip=198.41.200.73 2024-04-02T22:27:47Z INF Retrying connection in up to 1s connIndex=2 event=0 ip=198.41.200.73 2024-04-02T22:27:47Z ERR no more connections active and exiting 2024-04-02T22:27:47Z INF Tunnel server stopped 2024-04-02T22:27:47Z ERR icmp router terminated error="context canceled" 2024-04-02T22:27:47Z INF Metrics server stopped

Environment and versions

richardcavspain commented 2 months ago

Did you get anywhere with this? I have pretty much the same initial error - Failed to fetch features, default to disable error="lookup cfd-features.argotunnel.com on 127.0.0.11:53: read udp 127.0.0.1:41168->127.0.0.11:53: i/o timeout"

BrenoAquino commented 1 month ago

Same here

souja67 commented 1 month ago

Did you get anywhere with this? I have pretty much the same initial error - Failed to fetch features, default to disable error="lookup cfd-features.argotunnel.com on 127.0.0.11:53: read udp 127.0.0.1:41168->127.0.0.11:53: i/o timeout"

i have the same problem

richardcavspain commented 1 month ago

Still not working for me :-(

souja67 commented 1 month ago

Was able to fix my issue. Ended up being a DNS issue. My cloudflare tunnel container was unable to resolve DNS. I manually updated my DNS servers on my Synology to point to my router for primary and google (8.8.8.8) for my secondary. Hope this helps someone.

richardcavspain commented 1 month ago

Was able to fix my issue. Ended up being a DNS issue. My cloudflare tunnel container was unable to resolve DNS. I manually updated my DNS servers on my Synology to point to my router for primary and google (8.8.8.8) for my secondary. Hope this helps someone.

So in your case it was internal name resolution that was the issue? The HA box being unable to resolve argotunnel.com - right?

richardcavspain commented 1 month ago

For anyone having this issue, I managed to "fix" my installation, thanks to souja67 for the tip.

This was a DNS issue, I found that when I tried terminal inside HA, I could PING IP addresses on the Internet but not resolve their names, even though DNS, IP addresses and default gateways were setup correctly and I could access HA from within the local network. In the end (because I run HA in a VM) I backed it up, blew away the VM and set it up again from scratch, got HA running and then restored HA from backup and everything worked fine, confirming it was a VM issue, not a HA issue.

I hope this helps someone in the future :-)