cloudflare / cloudflared

Cloudflare Tunnel client (formerly Argo Tunnel)
https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/tunnel-guide
Apache License 2.0
8.95k stars 788 forks source link

Update fails on raspberry pi - context deadline exceeded #253

Open KommX opened 3 years ago

KommX commented 3 years ago

Since #249 got closed I want to create a new issue for the problem that was mentioned there and that some of us including me face: When I run sudo cloudflare update I get the error: failed to update cloudflared: context deadline exceeded (Client.Timeout or context cancellation while reading body)

More info:

cloudflared --version

cloudflared version 2020.10.0 (built 2020-10-06-1530 UTC)
cat /etc/os-release 

PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
ImaginaryElephant commented 3 years ago

I get this error too when I run sudo cloudflared update.

ERROR[2020-10-27T20:30:44Z] update check failed: context deadline exceeded (Client.Timeout or context cancellation while reading body) failed to update cloudflared: context deadline exceeded (Client.Timeout or context cancellation while reading body)

Previously I used to get

failed to update cloudflared: no release found

before the update to the API #249

I'm running cloudflared on a raspberry pi 3B cloudflared -v cloudflared version 2020.10.0 (built 2020-10-06-1530 UTC)

cat /etc/os-release PRETTY_NAME="Raspbian GNU/Linux 10 (buster)" NAME="Raspbian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=raspbian ID_LIKE=debian HOME_URL="http://www.raspbian.org/" SUPPORT_URL="http://www.raspbian.org/RaspbianForums" BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

tamhamilton commented 3 years ago

Same issue here.

cloudflared version 2020.10.0 (built 2020-10-06-1530 UTC)

PRETTY_NAME="Raspbian GNU/Linux 10 (buster)" NAME="Raspbian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=raspbian ID_LIKE=debian HOME_URL="http://www.raspbian.org/" SUPPORT_URL="http://www.raspbian.org/RaspbianForums" BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

tamhamilton commented 3 years ago

OK, fixed it. Well, more specifically, they fixed it...

This is how:

Roll back cloudflared to an older, stable, version: wget https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz tar -xvzf cloudflared-stable-linux-arm.tgz sudo systemctl stop cloudflared sudo cp ./cloudflared /usr/local/bin sudo chmod +x /usr/local/bin/cloudflared sudo systemctl start cloudflared cloudflared -v

Then update cloudflared (to 2020.10.2): sudo cloudflared update sudo systemctl restart cloudflared

Run the update command again, and you should get a sensible response about cloudflared being up to date

kosymodo commented 3 years ago

Thanks @tamhamilton - this worked a treat!

denisinla commented 3 years ago

sudo cloudflared update

Nope, still getting failed to update cloudflared: context deadline exceeded (Client.Timeout or context cancellation while reading body)

tamhamilton commented 3 years ago

Cloudflared definitely reporting v2020.10.2?

denisinla commented 3 years ago

Cloudflared definitely reporting v2020.10.2?

~Negative, it is still reporting 2020.10.0.~

Looks like I forgot a step sudo systemctl stop cloudflared. I can confirm sudo cloudflared update runs properly now.

Thanks @tamhamilton

KommX commented 3 years ago

OK, fixed it. Well, more specifically, they fixed it...

This is how:

Roll back cloudflared to an older, stable, version: wget https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz tar -xvzf cloudflared-stable-linux-arm.tgz sudo systemctl stop cloudflared sudo cp ./cloudflared /usr/local/bin sudo chmod +x /usr/local/bin/cloudflared sudo systemctl start cloudflared cloudflared -v

Then update cloudflared (to 2020.10.2): sudo cloudflared update sudo systemctl restart cloudflared

Run the update command again, and you should get a sensible response about cloudflared being up to date

I would say, that this is more of a workaround than a fix. What do the people do, that don't see this issue report here?

ddarnell1 commented 3 years ago

Thank you, the re-install stable then update method worked for me as well.

denisinla commented 3 years ago

Actually that wasn't a rollback. It's a re-install since the stable version is always the latest one.

tamhamilton commented 3 years ago

That's what I thought, but I checked the version after my initial steps and it didn't report 2020.10.2 (can't remember what it actually was though). Ran the update command, and it did update (rather than just report that cloudflared is at the latest version), and now reports 2020.10.2.

Anyhow, I apologise if my incorrect terminology offends anyone... Just wanted to share the steps that got me up and running properly.

ghost commented 3 years ago

Can also Replicate this. 2020.10.0 is what my version was. @KommX steps fixed my issue. Sounds like people will just need to deploy this fix for the updates to work again.

ferferga commented 3 years ago

The provided suggestions doesn't work for me. cloudflared -v output after running the commands:

cloudflared version 2020.10.2 (built 2020-10-21-1858 UTC)

tamhamilton commented 3 years ago

And what now happens when you run cloudflared update ?

ferferga commented 3 years ago

@tamhamilton failed to update cloudflared: Get "https://update.argotunnel.com?arch=arm&os=linux": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

tommetje commented 3 years ago

I was on cloudflared version 2020.10.2 (built 2020-10-21-1858 UTC) on a raspberry pi and could not update too (context deadline exceeded error).

Re-installing the older version enabled me to update successfully to cloudflared version 2020.11.3 (built 2020-11-11-1505 UTC)

ferferga commented 3 years ago

@tommetje But, after doing that, now in the latest version, are you able to update using cloudflared's built-in updater?

Because I know I'm able to update it replacing the binary. However, what I want to know is if I'll need to make an script for automatising the updating process, as I was relying in the built-in one for the automation.

Will the built-in updater be fixed at some point?

daltoniam commented 3 years ago

Howdy everyone. We just release 2020.11.3 which lengthen the timeout on the updater. The timeout was previously a bit short for some connections and thus the timeout error. Updating from the latest version forward should work as expected. A possible work without doing a manual upgrade/replace would to downgrade to the older updater and then upgrade to the latest version.

When running version 2020.10.0 run these commands:

cloudflared update --version=2020.9.3 --force
cloudflared update

This will force a downgrade to version 2020.9.3 with the old updater that will then update to the latest version 2020.11.3

KommX commented 3 years ago

@daltoniam Unfortunately the downgrade doesn't work either.

sudo cloudflared update --version=2020.9.3 --force

INFO[2020-11-11T20:42:19+01:00] cloudflared is set to upgrade to the latest publish version regardless of the current version
failed to update cloudflared: context deadline exceeded (Client.Timeout or context cancellation while reading body)
tommetje commented 3 years ago

@ferferga After running the update command, currently I am told that cloudflared is up to date.

I have a 1000mbit connection and I too got the timeout error message before I re-installed cloudflared, the timeout must have been set very short 😃

ferferga commented 3 years ago

@daltoniam Pulling the latest release from GitHub and checking for updates worked great! Thank you very much!

@KommX You musy grab the latets release from the releases tab and replace it in your system.

peloncano commented 3 years ago

OK, fixed it. Well, more specifically, they fixed it...

This is how:

Roll back cloudflared to an older, stable, version: wget https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz tar -xvzf cloudflared-stable-linux-arm.tgz sudo systemctl stop cloudflared sudo cp ./cloudflared /usr/local/bin sudo chmod +x /usr/local/bin/cloudflared sudo systemctl start cloudflared cloudflared -v

Then update cloudflared (to 2020.10.2): sudo cloudflared update sudo systemctl restart cloudflared

Run the update command again, and you should get a sensible response about cloudflared being up to date

This worked well for me. Thanks.

tamhamilton commented 3 years ago

The forced downgrade mentioned by @daltoniam didn't work for me either, attempting to go from 2020.10.2 to 2020.11.3, and the straight update also failed. However the steps I used previously worked a charm, again. Hopefully 2020.11.3 has cleared up the issue finally.

tamhamilton commented 3 years ago

Successfully updated cloudflared to 2020.11.4 from 2020.11.3 using the automatic updater; seems like the fix in .3 is functional!

tommetje commented 3 years ago

Same here, thanks for the fix!

KommX commented 3 years ago

@KommX You musy grab the latets release from the releases tab and replace it in your system.

I understand that this would be a workaround. But as I stated before, that's not a proper fix, since all the affected people out there would have to have knowledge about this workaround.

tamhamilton commented 3 years ago

Still intermittently getting this issue, actually. Just updated to 2020.11.10 successfully now, but was getting the error earlier today with 2020.11.9. Is it just 'internet weather'?

t-reyerf commented 3 years ago

OK, fixed it. Well, more specifically, they fixed it...

This is how:

Roll back cloudflared to an older, stable, version: wget https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz tar -xvzf cloudflared-stable-linux-arm.tgz sudo systemctl stop cloudflared sudo cp ./cloudflared /usr/local/bin sudo chmod +x /usr/local/bin/cloudflared sudo systemctl start cloudflared cloudflared -v

Then update cloudflared (to 2020.10.2): sudo cloudflared update sudo systemctl restart cloudflared

Run the update command again, and you should get a sensible response about cloudflared being up to date

I stumbled about this error because i had to restart my local dns resolver and cloudflared stoped working after that.

If you find yourself in the situation that you are running an older version on Debian/Ubuntu, you can use the following to get things going again:

wget https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-amd64.deb sudo systemctl stop cloudflared dpkg -i cloudflared-stable-linux-amd64.deb sudo systemctl start cloudflared

I was raised to V2020-12 after that and everything is working now again.

DFlexy commented 3 years ago

Same problem here need help

cloudflared version 2021.3.1 (built 2021-03-15-1925 UTC)

Mar 21 14:54:32 raspberrypi cloudflared[516]: 2021-03-21T17:54:32Z ERR failed to connect to an HTTPS backend "https://94.140.15.15/dns-query" error="failed to perform an HTTPS request: Post \"https://94.140.15.15/dns-query\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Mar 21 14:54:32 raspberrypi cloudflared[516]: 2021-03-21T17:54:32Z ERR failed to connect to an HTTPS backend "https://9.9.9.9/dns-query" error="failed to perform an HTTPS request: Post \"https://9.9.9.9/dns-query\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Mar 21 14:54:32 raspberrypi cloudflared[516]: 2021-03-21T17:54:32Z ERR failed to connect to an HTTPS backend "https://9.9.9.9/dns-query" error="failed to perform an HTTPS request: Post \"https://9.9.9.9/dns-query\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Mar 21 14:54:32 raspberrypi cloudflared[516]: 2021-03-21T17:54:32Z ERR failed to connect to an HTTPS backend "https://9.9.9.9/dns-query" error="failed to perform an HTTPS request: Post \"https://9.9.9.9/dns-query\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Mar 21 14:54:32 raspberrypi cloudflared[516]: 2021-03-21T17:54:32Z ERR failed to connect to an HTTPS backend "https://9.9.9.9/dns-query" error="failed to perform an HTTPS request: Post \"https://9.9.9.9/dns-query\": context deadline exceeded" Mar 21 14:54:32 raspberrypi cloudflared[516]: 2021-03-21T17:54:32Z ERR failed to connect to an HTTPS backend "https://9.9.9.9/dns-query" error="failed to perform an HTTPS request: Post \"https://9.9.9.9/dns-query\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Mar 21 14:54:32 raspberrypi cloudflared[516]: 2021-03-21T17:54:32Z ERR failed to connect to an HTTPS backend "https://9.9.9.9/dns-query" error="failed to perform an HTTPS request: Post \"https://9.9.9.9/dns-query\": context deadline exceeded" Mar 21 14:54:32 raspberrypi cloudflared[516]: 2021-03-21T17:54:32Z ERR failed to connect to an HTTPS backend "https://9.9.9.9/dns-query" error="failed to perform an HTTPS request: Post \"https://9.9.9.9/dns-query\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Mar 21 14:54:33 raspberrypi cloudflared[516]: 2021-03-21T17:54:33Z ERR failed to connect to an HTTPS backend "https://94.140.15.15/dns-query" error="failed to perform an HTTPS request: Post \"https://94.140.15.15/dns-query\": context deadline exceeded" Mar 21 14:54:33 raspberrypi cloudflared[516]: 2021-03-21T17:54:33Z ERR failed to connect to an HTTPS backend "https://94.140.15.15/dns-query" error="failed to perform an HTTPS request: Post \"https://94.140.15.15/dns-query\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"

tamhamilton commented 3 years ago

What version of cloudflared are you running? There are a couple of workarounds further up this thread...

ferferga commented 3 years ago

@DFlexy This issue is for that message appearing while running cloudflare update, which is not the exact same circumstance that you're having.

You will probably have better luck opening a new issue for that, as this is completely unrelated to the original issue.

ouwenxxing commented 3 years ago

I think there's no issues for the cloudflared itself but the upstream server. You can chose the DOH server regionally. I tried alidns and it works fine me. --upstream https://dns.alidns.com/dns-query