Segmentation fault on raspberry pi 2 model b

daniloncmayo commented 6 years ago

File downloaded from https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz

pi@raspberrypi:~ $ ./cloudflared -v
Segmentation fault

pi@raspberrypi:~ $ uname -a 
Linux raspberrypi 4.14.52+ #1123 Wed Jun 27 17:05:32 BST 2018 armv6l GNU/Linux
pi@raspberrypi:~ $ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 9.4 (stretch)
Release:    9.4
Codename:   stretch
pi@raspberrypi:~ $ cat /proc/cpuinfo 
processor   : 0
model name  : ARMv6-compatible processor rev 7 (v6l)
BogoMIPS    : 697.95
Features    : half thumb fastmult vfp edsp java tls 
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part    : 0xb76
CPU revision    : 7

Hardware    : BCM2835
Revision    : 000e
pi@raspberrypi:~ $ sha256sum cloudflared
b3730fd14bc7306b09eafefcef10025aca3f2e94a6059952426a5341ab6e4045  cloudflared
pi@raspberrypi:~ $ file cloudflared
cloudflared: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=adf2825c51e543d9a36dea416573b70eeaa2ac8a, not stripped

ghost commented 6 years ago

Same on Raspberry Pi Zero and Raspberry Pi Zero W.

Pulikutti commented 6 years ago

Same on Pi Zero W. reverted to 2018.6.2

Program received signal SIGSEGV, Segmentation fault. 0x00060ea0 in runtime.moduledataverify1 (datap=0x36333b31) at /usr/local/go/src/runtime/symtab.go:517

xyzulu commented 6 years ago

Exactly the same issue as well. It's also being reported elsewhere. Reverting to 2018.7.2 works..

the issue is 2018.7.3

ghost commented 6 years ago

@xyzulu how do you revert to 2018.7.2? I can't find the binaries for download.

xyzulu commented 6 years ago

Here: https://bin.equinox.io/a/4SUTAEmvqzB/cloudflared-2018.7.2-linux-arm.tar.gz

angussidney commented 6 years ago

I have also reproduced this issue on my Raspberry Pi Model 1B (rev 2). Reverting to 2018.7.2 as xyzulu suggested works.

rix1337 commented 6 years ago

Same issue, same device

cvocvo commented 6 years ago

+1 for same issue on a RPi Model B; 2018-7.2 works but I keep randomly running into this (https://github.com/cloudflare/cloudflared/issues/23) same error with 2018-7.2. failed to connect to an HTTPS backend and then have to reboot the Pi.

Would love to get this fixed to try a later version of Cloudflared that hopefully resolves that problem :)

dullroar commented 6 years ago

+1 on all of the above, including falling back to 2018-7.2 works.

tconnz commented 6 years ago

Same issue here on RPi B - Segmentation Fault.

EDIT- 2018-7.2 only works if copied to usr/local/bin and then chmod +x is run.

CBielstein commented 6 years ago

Same issue with a Raspberry Pi model A. Falling back to 2018-7.2 worked without any further steps.

silvether commented 6 years ago

+1 Can repro on Raspberry Pi 1 B+ with Raspbian Minimal

jmvermeulen commented 6 years ago

Same here on Pi 2.

Chlorus commented 6 years ago

+1 for a Raspberry Pi 1 Model B+, haven't tried reverting to an older version yet.

double-fault commented 5 years ago

Falling back to 2018-7.2 does work on a Raspberry Pi Zero W.

joehillen commented 5 years ago

I ended up building from source on my Pi Zero W, and it works like a charm. Had to add an extra swapfile in order for the build process to have enough RAM.

I suspect cloudflare's ARM build instance is to blame.

ravvle commented 5 years ago

I also had this issue on the Pi Zero W, building from source fixed this for me. I had to add extra swap space to compile properly. Thanks @joehillen

Note: I compiled using Go 1.11.5. I noticed the build config seems to use the 1.9 stream which doesn't receive patches any more.

dsadsa897897r commented 5 years ago

What command is needed to compile using Go?

sudo apt-get install golang
export GOPATH=$HOME/go
go install github.com/cloudflare/cloudflared/cmd/cloudflared

sssilver commented 5 years ago

Hi,

The next cloudflared release will be built with Go 1.11.

hexdra commented 5 years ago

Same error on Pi B Version 2019.2.1. 2018.7.2 works.

ArniDagur commented 5 years ago

@sssilver Since there has been a new release, does that mean this issue is fixed?

timothybrown commented 5 years ago

@sssilver Since there has been a new release, does that mean this issue is fixed?

Does not appear to have been fixed. I tried the current release today and it’s still segfaulting.

Trikolon commented 5 years ago

cloudflared version 2019.4.1 (built 2019-04-19-2152 UTC) (official binary download)) Works on Raspberry Pi 3b+

cleanev commented 5 years ago

Same issue on RPiZeroW with latest Rasberry Pi Stretch with all updates and name showing - Linux ZeroPiHole-I 4.19.42+ #1219 Tue May 14 21:16:38 BST 2019 armv6l GNU/Linux

Latest version 2019.4.1 only works with RPi3B+

pi@ZeroPiHole-I:~ $ dig yahoo.com

; <<>> DiG 9.10.3-P4-Raspbian <<>> yahoo.com ;; global options: +cmd ;; connection timed out; no servers could be reached pi@ZeroPiHole-I:~ $ dig @127.0.0.1 -p 5053 yahoo.com

; <<>> DiG 9.10.3-P4-Raspbian <<>> @127.0.0.1 -p 5053 yahoo.com ; (1 server found) ;; global options: +cmd ;; connection timed out; no servers could be reached pi@ZeroPiHole-I:~ $ ping 1.1.1.1 PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data. 64 bytes from 1.1.1.1: icmp_seq=1 ttl=55 time=23.9 ms 64 bytes from 1.1.1.1: icmp_seq=2 ttl=55 time=26.9 ms ^C --- 1.1.1.1 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1000ms rtt min/avg/max/mdev = 23.930/25.415/26.900/1.485 ms

Name resolution with 2018.7.2 does not work at all

I referenced this article when setting up DoH - https://docs.pi-hole.net/guides/dns-over-https/

billthefarmer commented 5 years ago

I think the segfault issue is due to released cloudflared being built on a later ARM processor, or cross compiled. The latest current version (2019.5.0) works fine on a Pi 3 B+, whether it stops working after a while (#23), I don't know, didn't test it for long enough, but segfault on a Pi 1B (BCM2835). I installed go1.12.5 linux/arm on the Pi 1B and built from source, release 2019.5.0 plus a few commits (babcd9f), this appears to work, but stops working after a while (#23), it appears to run out of file handles.

Jun  3 06:33:14 jennifer cloudflared[261]: time="2019-06-03T06:33:14Z" level=error msg="failed to connect to an HTTPS backend \"https://1.1.1.1/dns-query\"" error="failed to perform an HTTPS request: Post https://1.1.1.1/dns-query: dial tcp 1.1.1.1:443: socket: too many open files"
Jun  3 06:33:14 jennifer cloudflared[261]: time="2019-06-03T06:33:14Z" level=error msg="failed to connect to an HTTPS backend \"https://1.0.0.1/dns-query\"" error="failed to perform an HTTPS request: Post https://1.0.0.1/dns-query: dial tcp 1.0.0.1:443: socket: too many open files"
Jun  3 06:33:14 jennifer cloudflared[261]: time="2019-06-03T06:33:14Z" level=error msg="failed to connect to an HTTPS backend \"https://1.1.1.1/dns-query\"" error="failed to perform an HTTPS request: Post https://1.1.1.1/dns-query: dial tcp 1.1.1.1:443: socket: too many open files"
Jun  3 06:33:14 jennifer cloudflared[261]: time="2019-06-03T06:33:14Z" level=error msg="failed to connect to an HTTPS backend \"https://1.0.0.1/dns-query\"" error="failed to perform an HTTPS request: Post https://1.0.0.1/dns-query: dial tcp 1.0.0.1:443: socket: too many open files"

Update: Having looked at the Makefile, I built version 2019.5.0 using it...

pi@jennifer:~ $ cd gocode/src/github.com/cloudflare/cloudflared/
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ git checkout 2019.5.0
Note: checking out '2019.5.0'.
...
HEAD is now at 4bff1ef... Release 2019.5.0
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ export PATH=$PATH:/usr/local/go/bin
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ export GOPATH=~/gocode
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ go clean
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ make cloudflared
go build -v -ldflags='-X "main.Version=2019.5.0" -X "main.BuildTime=2019-06-03-1717 UTC"' github.com/cloudflare/cloudflared/cmd/cloudflared
pi@jennifer:~/gocode/src/github.com/cloudflare/cloudflared $ ./cloudflared -v
cloudflared version 2019.5.0 (built 2019-06-03-1717 UTC)

This version may be ok, it's now been running for several hours including a Pi reboot and a router reboot.

billthefarmer commented 5 years ago

Version 2019.5.0 built with go1.12.5 on a Pi 1B as in previous post eventually failed due to running out of file handles after about a week...

Jun 10 08:27:44 jennifer cloudflared[1286]: time="2019-06-10T08:27:44Z" level=error msg="failed to connect to an HTTPS backend \"https://1.1.1.1/dns-query\"" error="failed to perform an HTTPS request: Post https://1.1.1.1/dns-query: dial tcp 1.1.1.1:443: socket: too many open files"
Jun 10 08:27:44 jennifer cloudflared[1286]: time="2019-06-10T08:27:44Z" level=error msg="failed to connect to an HTTPS backend \"https://1.0.0.1/dns-query\"" error="failed to perform an HTTPS request: Post https://1.0.0.1/dns-query: dial tcp 1.0.0.1:443: socket: too many open files"

lightswitch05 commented 5 years ago

Just checking in that the issue continues with version 2019.6.0

billthefarmer commented 5 years ago

The latest version from https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz gives an instant segfault on a Pi 1B. I can't tell the version because it won't run.

pi@jennifer:~/cd $ tar xzvf cloudflared-stable-linux-arm.tgz
cloudflared
pi@jennifer:~/cd $ ./cloudflared
Segmentation fault

Update: Version 2019.6.0 builds and runs OK on a PI 1B using go1.12.5 linux/arm, downloading an amazing long list of stuff from various sources.


pi@jennifer:~/go/src/github.com/cloudflare/cloudflared $ git checkout 2019.6.0
Note: checking out '2019.6.0'.
...
HEAD is now at acd17f6... Release 2019.6.0
pi@jennifer:~/go/src/github.com/cloudflare/cloudflared $ export GOPATH=~/go
pi@jennifer:~/go/src/github.com/cloudflare/cloudflared $ export PATH=$PATH:/usr/local/go/bin
pi@jennifer:~/go/src/github.com/cloudflare/cloudflared $ make cloudflared
go build -v -ldflags='-X "main.Version=2019.6.0" -X "main.BuildTime=2019-06-21-1621 UTC"' github.com/cloudflare/cloudflared/cmd/cloudflared
... Amazing long list of stuff from various sources ...
pi@jennifer:~/go/src/github.com/cloudflare/cloudflared $ ./cloudflared -v
cloudflared version 2019.6.0 (built 2019-06-21-1621 UTC)

cleanev commented 5 years ago

Somewhere I found a tgz file that was compiled for 32 bit Pi, I am assuming those are zero, zero w, a and some older ones? Upon checking the version I have is 2019.5.0 that works perfectly fine on ZeroW. Can’t seem to find the original post or url that hosted the file, however I do have a copy that I can share. I am not sure how to attach it here. Let me know if someone has a website where I can upload

billthefarmer commented 5 years ago

I have on soak test on a Pi 1B an old version I got from here: https://web.archive.org/web/20180419005946/https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz, linked from here: https://web.archive.org/web/20180524132333/https://developers.cloudflare.com/argo-tunnel/downloads, which is version 2018.4.6. It's third choice, so it may take a while to fail if it's going to.

A recent version built from source using a recent Go compiler seems to fail after a while as above.

pmarella2 commented 5 years ago

Anyone have any method to get it working on RPi Zero? I tried downloading different versions and they work without giving a segmentation fault. But, if I restart my RPi or after a while, it just stops working.

lightswitch05 commented 5 years ago

@pmarella2 disable auto-update in your config yaml: no-autoupdate: true

maximal commented 5 years ago

Same issue.

cemtes commented 5 years ago

Recompiled binaries are available here https://hobin.ca/cloudflared/ . I can confirm that 2019.8.1 runs fine on my Raspberry Pi Zero. I installed using the normal method in the cloudflared docs but replaced

wget https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz

with

wget https://hobin.ca/cloudflared/releases/2019.8.1/cloudflared_2019.8.1_arm.tar.gz

Hope this helps others.

EDIT: I'm just passing along a solution I found, I had no part in generating these files and don't take any credit for them.

timothybrown commented 5 years ago

@cemtes Thanks! Works great on my Pi B+!

billthefarmer commented 5 years ago

I found that any version of cloudflared flaky on a Pi 1B, either an old build or a rebuilt recent version, in that it stops working after a while. I then tried DNSCrypt-Proxy, which also stopped working after a while. The common factor appears to be that they are both written in Go. I then tried unbound, which works and has the advantage that it's already in the Raspbian repository.

mheland commented 5 years ago

@cemtes Thank you, stable on Pi 1B

pi@raspberrypi:/var/log $ cloudflared --v
cloudflared version 2019.8.1 (built 2019-08-07-0625 UTC)

tckb commented 5 years ago

@cemtes thanks this one works

meliurwen commented 5 years ago

I see many people started to use the binaries linked by @cemtes and I'm a little worried. The author of those binaries (reddit username probably DTHCND) didn't provide documentation to reproduce its build process.

Please be really careful and if possible restrain from using untrusted or unknown binary blobs.

Especially for tools like cloudflared, where one of its core features is privacy and distrust of the actors handling/forwarding your DNS requests between you and the Cloudflare's servers. Using those binaries, unless for whatever reason you trust this person, makes the use of this tool almost (if not completely) pointless.

mheland commented 5 years ago

Agree @meliurwen - I switched to DNS-crypt Proxy, easy to install / configure, light on resources and can run DoH with cloudflared. This guide is up to date: https://itchy.nl/raspberry-pi-3-with-openvpn-pihole-dnscrypt

timothybrown commented 5 years ago

@meliurwen For what it’s worth, this was my first thought as well, so I’ve been keeping an eye on the system I installed them on over the last two weeks. So far they haven’t made a single network request outside of the Cloudflare DNS servers and they’re not secretly storing requests to disk. Obviously this is not a through audit, but based on that and a bit of background on the person who made them available, I feel they’re safe.

Obviously everyone should make up their own mind and do their own testing (for all you know I could be in on it) and I do agree with the point you’re making. If you do use these I would recommend setting up a system to monitor outbound network traffic so you can make sure they’re not doing anything nefarious.

Limeth commented 5 years ago

It would be great if the author of those binaries shared how they got it to compile. I tried compiling it for my Raspberry Pi 1B, with my limited knowledge of golang, to no avail.

billthefarmer commented 5 years ago

It's possible to compile it on a Pi 1B, you need to install a recent version of the Go compiler, clone the cloudflared repository, checkout the release you want, and use the Makefile. See my comment above https://github.com/cloudflare/cloudflared/issues/38#issuecomment-504482367. The version from Raspbian may not be recent enough.

maximal commented 5 years ago

@billthefarmer, which version of Go do we need?

billthefarmer commented 5 years ago

I used go1.12.5 linux/arm. I think that was the latest at the time.

Edit: You will probably need to increase the swap size.

maximal commented 5 years ago

Confirmed. It compiles and works successfully when is built by Go of latest version.

aaa118 commented 5 years ago

Recompiled binaries are available here https://hobin.ca/cloudflared/ . I can confirm that 2019.8.1 runs fine on my Raspberry Pi Zero. I installed using the normal method in the cloudflared docs but replaced

wget https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz

with

wget https://hobin.ca/cloudflared/releases/2019.8.1/cloudflared_2019.8.1_arm.tar.gz

Hope this helps others.

EDIT: I'm just passing along a solution I found, I had no part in generating these files and don't take any credit for them.

This is the only solution which worked for me on pi zero

hobindar commented 5 years ago

It would be great if the author of those binaries shared how they got it to compile. I tried compiling it for my Raspberry Pi 1B, with my limited knowledge of golang, to no avail.

@Limeth Author of those binaries here. Building the binaries for ARMv6 is rather straightforward. You just have to make sure you have a C cross-compiler for ARM, such as gcc-arm-linux-gnueabi for Debian-based systems. Then set the following environment variables...

GOOS=linux
GOARCH=arm
GOARM=6
CGO_ENABLED=1
CC=arm-linux-gnueabi-gcc # This should be your ARM C cross-compiler

Now just build it...

go build -v "-ldflags=-X 'main.Version=${version}' -X 'main.BuildTime=${date}'" github.com/cloudflare/cloudflared/cmd/cloudflared

...where ${version} is the version of Cloudflared and ${date} is the current time.

Then just bundle the resulting executable however you want.

Thanks to @meliurwen for pointing me here from reddit.

Edit: Rather than building it with go build ..., you should be able to use their Makefile as well. I'm building it the way described above, however. Also worth noting, if you build on your Pi itself, you could probably skip the environment variables (the defaults target your build machine).

Last edit: Quick word on reproducibility, in case anyone is trying to reproduce these binaries byte-for-byte for the sake of auditing. Go builds are only reproducible if built with the same GOPATH and GOROOT. So those values for me (as of version 2019.8.3) are...

GOPATH=/home/hobindar/go
GOROOT=/usr/lib/go-1.13

You'll obviously also need to use the same version and date string that I used. You can get this string by running cloudflared version, which will print out a line like cloudflared version foo (built bar). In this example, foo is the version string and bar is the date string.

More information about byte-for-byte reproducibility in Go can be found here: https://blog.filippo.io/reproducing-go-binaries-byte-by-byte/

Last edit for real: The following (mentioned in an earlier comment) will grab version 2019.8.1...

wget https://hobin.ca/cloudflared/releases/2019.8.1/cloudflared_2019.8.1_arm.tar.gz

You could, however, grab the latest (regardless of version) using...

wget https://hobin.ca/cloudflared/latest?type=tar

...replacing "tar" with the package type of your choice (i.e. "deb", "rpm", or "tar").

This more closely mimics the behavior of the equinox link in their documentation (by grabbing latest).

Chlorus commented 5 years ago

FWIW, I got tired of waiting for a fix upstream & just switched to dnscrypt-proxy, which supports DoH now & is a lot more flexible in the config.

nolanlawson commented 5 years ago

Update: I got it working! See complete steps below. :slightly_smiling_face:

Original comment, click to view

I'm not able to follow the instructions from @hobindar. When I build the `cloudflared` binary and run it on my Raspberry Pi Zero, it prints: -bash: ./cloudflared: cannot execute binary file: Exec format error I'm trying to build this on Ubuntu 18.04 amd64. Here are my steps to reproduce: - Install Go for Ubuntu as described [on the Golang wiki](https://github.com/golang/go/wiki/Ubuntu) - `go version` prints `go version go1.13.3 linux/amd64` - ` sudo apt install gcc-arm-linux-gnueabi` - set the following env variables: ```sh version=2018.12.1 date=$(date) GOOS=linux GOARCH=arm GOARM=6 CGO_ENABLED=1 CC=arm-linux-gnueabi-gcc ``` Download the cloudflared source and put it where Go can find it: ```sh git clone git@github.com:cloudflare/cloudflared.git ~/go/src/github.com/cloudflare/cloudflared ``` Build: ```sh go build -v "-ldflags=-X 'main.Version=${version}' -X 'main.BuildTime=${date}'" github.com/cloudflare/cloudflared/cmd/cloudflared ``` But when I copy this to my Pi and run it, I get the error above. Hopefully someone can use my instructions, tweak them slightly, and figure out how to build this correctly. :slightly_smiling_face:

On Ubuntu 18.04, here's how I managed to compile cloudflared for my Raspberry Pi Zero W:

Install Go for Ubuntu as described on the Golang wiki.

Check go version, it should print go version go1.13.3 linux/amd64.

Run the following:

# install required CC
sudo apt install gcc-arm-linux-gnueabi

# clone cloudflared source where Go can find it
git clone git@github.com:cloudflare/cloudflared.git ~/go/src/github.com/cloudflare/cloudflared

# build (will create cloudflared binary in current directory)
version=2018.12.1 date=$(date) GOOS=linux GOARCH=arm GOARM=6 CGO_ENABLED=1 CC=arm-linux-gnueabi-gcc go build -v "-ldflags=-X 'main.Version=${version}' -X 'main.BuildTime=${date}'" github.com/cloudflare/cloudflared/cmd/cloudflared

Then copy the cloudflared file to your Pi (e.g. with SSH) and run ./cloudflared -v to make sure it's working.

My mistake before was that I forgot to export the env variables, or to pass them in on the same line where I run the go build command. So it was just building for Linux amd64 instead of for the Pi's architecture. Thanks @hobindar!

cloudflare / cloudflared

Segmentation fault on raspberry pi 2 model b #38