search

gravitational / teleport

The easiest, and most secure way to access and protect all of your infrastructure.
https://goteleport.com
GNU Affero General Public License v3.0
17.04k stars 1.71k forks source link

ARM binaries do not work on 64-bit ARM ubuntu (Raspberry Pi 4) #3383

Closed gclawes closed 3 years ago

gclawes commented 4 years ago

What happened: ARM binaries do not run on 64-bit Ubuntu running on a Raspberry Pi 4

What you expected to happen: Binaries can be exectued

How to reproduce it (as minimally and precisely as possible):

ubuntu@pi01:~/teleport$ ls -lah
total 95M
drwxrwxr-x  3 ubuntu ubuntu 4.0K Feb 21 15:35 .
drwxr-xr-x  6 ubuntu ubuntu 4.0K Feb 26 00:30 ..
-rw-rw-r--  1 ubuntu ubuntu  60K Feb 21 15:35 CHANGELOG.md
-rw-rw-r--  1 ubuntu ubuntu 8.1K Feb 21 15:35 README.md
-rw-rw-r--  1 ubuntu ubuntu    7 Feb 21 15:35 VERSION
drwxrwxr-x 12 ubuntu ubuntu 4.0K Feb 21 15:35 examples
-rwxrwxr-x  1 ubuntu ubuntu  483 Feb 21 15:35 install
-rwxrwxr-x  1 ubuntu ubuntu  32M Feb 21 15:35 tctl
-rwxrwxr-x  1 ubuntu ubuntu  42M Feb 21 15:35 teleport
-rwxrwxr-x  1 ubuntu ubuntu  22M Feb 21 15:35 tsh
ubuntu@pi01:~/teleport$ cat VERSION
v4.2.3
ubuntu@pi01:~/teleport$ ./tsh
-bash: ./tsh: No such file or directory
ubuntu@pi01:~/teleport$ file tsh
tsh: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, Go BuildID=loIAEgzMzAh5YmfjehQr/m8eZoEzX4mavUHlw9Dvh/TsBtUbDo5EEM3df4dbY3/H-fKAJwbrHQAi_-y08vR, BuildID[sha1]=160a0d8fc7ddbff40885461220b3e379de40a4cc, stripped

Environment:

processor : 1 BogoMIPS : 108.00 Features : fp asimd evtstrm crc32 cpuid CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd08 CPU revision : 3

processor : 2 BogoMIPS : 108.00 Features : fp asimd evtstrm crc32 cpuid CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd08 CPU revision : 3

processor : 3 BogoMIPS : 108.00 Features : fp asimd evtstrm crc32 cpuid CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd08 CPU revision : 3

Hardware : BCM2835 Revision : c03112 Serial : 10000000fc316a97 Model : Raspberry Pi 4 Model B Rev 1.2



NOTE: This is related to my personal use of Teleport Community Edition, not my employer's use of Teleport Enterprise.
webvictim commented 4 years ago

Interesting. It looks like Ubuntu's support for 64-bit on the Pi 4 is quite recent. I'm running Raspbian buster on my own personal Pi 4 and Teleport works fine.

gus@artemis:~ $ uname -a
Linux artemis 4.19.97-v7l+ #1294 SMP Thu Jan 30 13:21:14 GMT 2020 armv7l GNU/Linux
gus@artemis:~ $ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
gus@artemis:~ $ cat /proc/cpuinfo
processor   : 0
model name  : ARMv7 Processor rev 3 (v7l)
BogoMIPS    : 108.00
Features    : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part    : 0xd08
CPU revision    : 3

processor   : 1
model name  : ARMv7 Processor rev 3 (v7l)
BogoMIPS    : 108.00
Features    : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part    : 0xd08
CPU revision    : 3

processor   : 2
model name  : ARMv7 Processor rev 3 (v7l)
BogoMIPS    : 108.00
Features    : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part    : 0xd08
CPU revision    : 3

processor   : 3
model name  : ARMv7 Processor rev 3 (v7l)
BogoMIPS    : 108.00
Features    : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part    : 0xd08
CPU revision    : 3

Hardware    : BCM2835
Revision    : b03111
Serial      : 10000000b0a6b070
Model       : Raspberry Pi 4 Model B Rev 1.1
gus@artemis:~ $ which tsh
/usr/local/bin/tsh
gus@artemis:~ $ tsh version
Teleport v4.2.0 git:v4.2.0-0-g3090806 go1.13.2

Our ARM build boxes are also armv7l or armhf (i.e. 32-bit), so it's unlikely that we'll have 64-bit ARM builds regularly available until we set up a 64-bit ARM build box.

I did a quick check on an AWS a1.large box running 64-bit Ubuntu and it does appear that you can build Teleport from source without any trouble, so that might be an option in the meantime. These are the steps I used:

sudo apt-get -y update && sudo apt-get -y install zip build-essential
wget https://dl.google.com/go/go1.13.2.linux-arm64.tar.gz
tar xvzf go1.13.2.linux-arm64.tar.gz
sudo mv go /usr/local/
echo "export PATH=/usr/local/go/bin:$PATH" >> ~/.bashrc
source ~/.bashrc
go get github.com/gravitational/teleport
cd ~/go/src/github.com/gravitational/teleport
git checkout tags/v4.2.3
make release

This puts teleport-v4.2.3-linux-arm64-bin.tar.gz in the ~/go/src/github.com/gravitational/teleport directory and then you can use this as desired.

gclawes commented 4 years ago

Raspbian is still a 32-bit userland, with the option to enable a 64-bit kernel on the Pi 4. Ubuntu is 64-bit userland and kernel.

gclawes commented 3 years ago

This is an issue on Manjaro Linux aarch64 as well

[gclawes@just-read-the-instructions teleport]$ cat /etc/os-release 
NAME="Manjaro-ARM"
ID=manjaro-arm
ID_LIKE=manjaro,arch
PRETTY_NAME="Manjaro ARM"
ANSI_COLOR="1;32"
HOME_URL="https://www.manjaro.org/"
SUPPORT_URL="https://forum.manjaro.org/c/manjaro-arm/"

[gclawes@just-read-the-instructions teleport]$ uname -a
Linux just-read-the-instructions 5.8.3-2-MANJARO-ARM #1 SMP Sat Aug 22 21:00:07 CEST 2020 aarch64 GNU/Linux
[gclawes@just-read-the-instructions teleport]$ file ./tsh
./tsh: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, Go BuildID=crhkFgAkhYoa_WcHNS2a/5xOrYJ9gAKmQ8UCmyzCX/BSfVolzsaA0q1gVvxNld/dWCl6K0GbCVKlg5avZVm, BuildID[sha1]=0183bfa0b1bf1c0cc18cfc845c5f31ebee36bfd7, stripped
[gclawes@just-read-the-instructions teleport]$ ./tsh
bash: ./tsh: No such file or directory
awly commented 3 years ago

https://github.com/gravitational/teleport/issues/1812 is the same issue and was reported earlier. Let's keep all the discussion there.

We need to set up dedicated aarch64 build servers (for linux, mac and maybe windows).