(Raspberry Pi) cannot execute binary: Exec format error

[julisch94]

Seems like the build doesn't work with Raspberry Pi (ARMv7, Raspbian Jessie).

How can I compile the sources to be able to run rqlite on my Raspberry Pi?

[otoolep]

@julisch94 -- you have included zero details on the build error, and your build process. I don't know how you expect people to help if you don't include more information.

[otoolep]

Unfortunately I have zero experience with Raspberry Pi, so I'm unlikely to be able to help. Once you provide more technical details, it might make sense to send a message to more general group, like the Go google group.

[julisch94]

I had done what's here under "Linux" and got the error above. By now, I have realized that I can't just use the supplied binaries, but instead I need to build from source to make rqlite work on Raspberry PI.

So I've done what's here and found out that

cd $GOPATH/bin/rqlited
go build
./rqlited ~/node.1

won't work, because $GOPATH/bin/rqlited is not a directory but a file instead. Usually you build source files from $GOPATH/src/ but unfortunately I don't know how to build $GOPATH/src/github.com/rqlite/rqlite as it contains the following directories and files:

CHANGELOG.md     appveyor.yml     cmd              gofmt.sh         tcp
CONTRIBUTING.md  auth             db               http             testdata
LICENSE          aws              disco            package.sh       vagrant_setup.sh
README.md        circle.yml       doc              store
Vagrantfile      cluster          doc.go           system_test

I want to build these source files with GOOS=linux GOARCH=arm to make it work on Raspberry PI, but I can't build the given folder $GOPATH/src/github.com/rqlite/rqlite. When trying to build that folder nothing happens and no binary file appears.

Which folder do I have to build to get the binaries for rqlite and rqlited? Since this is not a specific Raspberry PI problem but refers to building from source in general, I'd like to stay on this discussion if possible.

[otoolep]

Actually there was a typo in the build directions -- thanks for pointing that out. I have fixed it here: https://github.com/rqlite/rqlite/commit/bb89117c7c4242446cfcc11f34fdf3b69f61b154

Does that help?

[julisch94]

I am currently running > GOOS=linux GOARCH=arm go build -v . on both folders:

• $GOPATH/src/github.com/rqlite/rqlite/cmd/rqlite
• $GOPATH/src/github.com/rqlite/rqlite/cmd/rqlited

Actually, this seems to me to be the most reasonable option.

What it does: The first one, rqlite, seems to work. I can run the generated binary file on Raspberry PI. That looks fine. Whereas rqlited does not work. Running the command stated above doesn't generate any file. During the verbose build of rqlited, I can see some lines indicating an error regarding mattn/go-sqlite3:

(...)
text/template/parse
text/template
github.com/mattn/go-sqlite3
# github.com/mattn/go-sqlite3
../../../../mattn/go-sqlite3/sqlite3_go18.go:18: undefined: SQLiteConn
../../../../mattn/go-sqlite3/sqlite3_go18.go:26: undefined: SQLiteConn
../../../../mattn/go-sqlite3/sqlite3_go18.go:27: undefined: namedValue
../../../../mattn/go-sqlite3/sqlite3_go18.go:29: undefined: namedValue
../../../../mattn/go-sqlite3/sqlite3_go18.go:35: undefined: SQLiteConn
../../../../mattn/go-sqlite3/sqlite3_go18.go:36: undefined: namedValue
../../../../mattn/go-sqlite3/sqlite3_go18.go:44: undefined: SQLiteConn
../../../../mattn/go-sqlite3/sqlite3_go18.go:49: undefined: SQLiteConn
../../../../mattn/go-sqlite3/sqlite3_go18.go:54: undefined: SQLiteStmt
../../../../mattn/go-sqlite3/sqlite3_go18.go:63: undefined: SQLiteStmt
../../../../mattn/go-sqlite3/sqlite3_go18.go:36: too many errors
github.com/boltdb/bolt
regexp
crypto/x509
(...)

Any ideas if those errors affect the build process to exit unsuccessfully? I wonder why go build doesn't give any kind of error message. It keeps on running the build process as everything's good.

[otoolep]

Those are build errors specific to the SQLite library. I suggest you open a ticket on https://github.com/mattn/go-sqlite3 and reference this ticket. I didn't write that software.

[otoolep]

You might learn something from here: https://github.com/mattn/go-sqlite3/issues/242

[julisch94]

I finally figured out, how to do this and decided to let anyone facing the same issue benefit from my effort. This solution uses a Debian system running in a Docker container. Cross-compiling C programs (sqlite3) to ARM seems to work on Linux.

Hint: You'll need to have docker installed and running on your Mac.

Preparations on Host

1. create directory and download golang

mkdir ~/vol
cd ~/vol
curl -O https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz  

for more info see https://golang.org/doc/install for installation guide and https://golang.org/dl/ for appropriate packages or versions

2. pull debian jessie image and start with mounted directory

docker pull debian:jessie
docker run -it -v ~/vol:/volume debian:jessie

(docker container starts)

Preparations in Docker Container

1. Setup golang:

cd /volume
tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz
export PATH=$PATH:/usr/local/go/bin
cd
mkdir go
cd go
mkdir bin
mkdir pkg
mkdir src
export GOPATH=$PWD

2. install curl and git:

apt-get update
apt-get install curl -y
apt-get install git-core -y

3. install cross compiler:

echo "deb http://emdebian.org/tools/debian/ jessie main" > /etc/apt/sources.list.d/crosstools.list
curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add -
dpkg --add-architecture armhf
apt-get update
apt-get install crossbuild-essential-armhf -y

Will take a while.
for more info see https://wiki.debian.org/CrossToolchains#Installation

4. get rqlite

go get -u -t github.com/rqlite/rqlite/...

5a. build rqlite binary for arm

cd $GOPATH/src/github.com/rqlite/rqlite/cmd/rqlite
CC=arm-linux-gnueabihf-gcc GOOS=linux CGO_ENABLED=1 GOARCH=arm GOARM=7 go build -v .
cp rqlite /volume

5b. build rqlited binary for arm

cd $GOPATH/src/github.com/rqlite/rqlite/cmd/rqlited
CC=arm-linux-gnueabihf-gcc GOOS=linux CGO_ENABLED=1 GOARCH=arm GOARM=7 go build -v .
cp rqlited /volume

6. finally

You'll find both binaries in ~/vol on your host machine (Mac).

~/vol> ls
go1.8.3.linux-amd64.tar.gz     rqlited      rqlite

7. further do

• Transfer both files via scp to your Raspberry PI
• SSH connect to your Raspberry PI
• Run the files. It works fine!

[otoolep]

Great, thanks for following up @julisch94

[wil222]

Hi,

I'm wondering why you go through all those steps... I just ran into the same issue but I was able to compile rqlite directly on the raspberry just by downloading the sources and by running the relevant part of package.sh

It works like a charm while being quicker to compile

[otoolep]

OK, thanks for the update @wil222

[sonyarouje]

This is how I build for arm and arm64. I used a pi4 running rasbpian 64bit beta os Note: I haven't tested the 32bit binary yet. ARM64 binary works fine.

sudo apt-get install crossbuild-essential-armhf -y
git clone https://github.com/rqlite/rqlite
cd /home/pi/rqlite/cmd/rqlited
# build for arm
CC=arm-linux-gnueabihf-gcc GOOS=linux GOARCH=arm GOARM=7 CGO_ENABLED=1 go build -ldflags="-s -w" -o rqlited-arm main.go

# build for arm64
GOOS=linux GOARCH=arm64 go build -ldflags="-s -w" -o rqlited-arm64 main.go

[moving-digital]

This is how I build for arm and arm64. I used a pi4 running rasbpian 64bit beta os Note: I haven't tested the 32bit binary yet. ARM64 binary works fine.

sudo apt-get install crossbuild-essential-armhf -y
git clone https://github.com/rqlite/rqlite
cd /home/pi/rqlite/cmd/rqlited
# build for arm
CC=arm-linux-gnueabihf-gcc GOOS=linux GOARCH=arm GOARM=7 CGO_ENABLED=1 go build -ldflags="-s -w" -o rqlited-arm main.go

# build for arm64
GOOS=linux GOARCH=arm64 go build -ldflags="-s -w" -o rqlited-arm64 main.go

Hi @otoolep,

Thanks for your amazing rqlite software. I wanted to use it just now with a RaspberryPi 4B with 32bit OS (Original Raspbian Buster 10), but I am also facing the same issue as described by the OP.

Some more info here below:

pi@raspberrypi:~ $ curl -L https://github.com/rqlite/rqlite/releases/download/v6.0.0/rqlite-v6.0.0-linux-amd64 rqlite-v6.0.0-linux-amd64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   638  100   638    0     0   2064      0 --:--:-- --:--:-- --:--:--  2071
100 17.3M  100 17.3M    0     0  8579k      0  0:00:02  0:00:02 --:--:-- 11.2M
pi@raspberrypi:~ $ tar xvfz rqlite-v6.0.0-linux-amd64.tar.gz
rqlite-v6.0.0-linux-amd64/
rqlite-v6.0.0-linux-amd64/rqlite
rqlite-v6.0.0-linux-amd64/rqbench
rqlite-v6.0.0-linux-amd64/rqlited
pi@raspberrypi:~ $ cd rqlite-v6.0.0-linux-amd64
**pi@raspberrypi:~/rqlite-v6.0.0-linux-amd64 $ ./rqlited ~/node.1
-bash: ./rqlited: cannot execute binary file: Exec format error**
pi@raspberrypi:~/rqlite-v6.0.0-linux-amd64 $ sudo su
**root@raspberrypi:/home/pi/rqlite-v6.0.0-linux-amd64# ./rqlited ~/node.1
bash: ./rqlited: cannot execute binary file: Exec format error**

Hello @sonyarouje,

Better late than never: I've just tested your piece of code for the ARM build only (Not the 64 bits one). And it works like a charm. You're a genius! Now, this is the screen I'm getting:

root@raspberrypi:/home/pi/Documents/rqlite/rqlite/cmd/rqlited# ./rqlited-arm -node-id 1 ~/node.1

            _ _ _
           | (_) |
  _ __ __ _| |_| |_ ___
 | '__/ _  | | | __/ _ \   The lightweight, distributed
 | | | (_| | | | ||  __/   relational database.
 |_|  \__, |_|_|\__\___|
         | |               www.rqlite.com
         |_|

[rqlited] 2021/06/12 16:15:46 rqlited starting, version 6, commit unknown, branch unknown
[rqlited] 2021/06/12 16:15:46 go1.16.5, target architecture is arm, operating system target is linux
[rqlited] 2021/06/12 16:15:46 launch command: ./rqlited-arm -node-id 1 /root/node.1
[mux] 2021/06/12 16:15:46 received handler registration request for header 1
[mux] 2021/06/12 16:15:46 received handler registration request for header 2
[cluster] 2021/06/12 16:15:46 service listening on 127.0.0.1:4002
[rqlited] 2021/06/12 16:15:46 no preexisting node state detected in /root/node.1, node may be bootstrapping
[rqlited] 2021/06/12 16:15:46 no join addresses set
[store] 2021/06/12 16:15:46 opening store with node ID 1
[store] 2021/06/12 16:15:46 ensuring directory at /root/node.1 exists
[mux] 2021/06/12 16:15:46 mux serving on 127.0.0.1:4002, advertising 127.0.0.1:4002
[store] 2021/06/12 16:15:46 0 preexisting snapshots present
[store] 2021/06/12 16:15:46 first log index: 0, last log index: 0, last command log index: 0:
2021-06-12T16:15:46.994+0800 [INFO]  raft: initial configuration: index=0 servers=[]
[store] 2021/06/12 16:15:46 executing new cluster bootstrap
2021-06-12T16:15:46.995+0800 [INFO]  raft: entering follower state: follower="Node at 127.0.0.1:4002 [Follower]" leader=
2021-06-12T16:15:48.350+0800 [WARN]  raft: heartbeat timeout reached, starting election: last-leader=
2021-06-12T16:15:48.350+0800 [INFO]  raft: entering candidate state: node="Node at 127.0.0.1:4002 [Candidate]" term=2
2021-06-12T16:15:48.358+0800 [INFO]  raft: election won: tally=1
2021-06-12T16:15:48.358+0800 [INFO]  raft: entering leader state: leader="Node at 127.0.0.1:4002 [Leader]"
[store] 2021/06/12 16:15:48 waiting for up to 2m0s for application of initial logs
[rqlited] 2021/06/12 16:15:48 store has reached consensus
[http] 2021/06/12 16:15:48 service listening on 127.0.0.1:4001
[rqlited] 2021/06/12 16:15:48 node is ready

@otoolep do you think it would be worth looking at supporting ARM and ARM64 natively with Rqlite, without having to cross-compile? I think a lot of people nowadays are using Rpi for many applications, do you think it would make sense?

[virtadpt]

You're running a 32-bit build of RaspberryOS but are trying to execute a 64-bit build of rqlite. Without 32-to-64 bit compatibility libraries, that won't work.

[otoolep]

The issue is I don't have access to an ARM system, so anything I build I couldn't test.

[NerdyShawn]

Just to update those interested, multi-arch builds are now available today on the dockerhub image, thanks! :partying_face:

• https://github.com/rqlite/rqlite/releases/tag/v8.24.9