SeedSigner / seedsigner

Use an air-gapped Raspberry Pi Zero to sign for Bitcoin transactions! (and do other cool stuff)
MIT License
721 stars 168 forks source link

Manual installation on Debian 11 #431

Open hax0rbana-adam opened 1 year ago

hax0rbana-adam commented 1 year ago

Build instructions fail on Debian 11 (there's a separate ticket about Debian 12 #430).

It'd be good if a core contributor could lay out what they'd like the roadmap to be in order to fix this (or how developer should set up an environment once Debian 10 is not longer supported).

This failure on Debian 11 was reported over a year ago in #149, but that was closed without being fixed nor being labeled as "won't fix", so it's not clear what the status is here. Buster (Debian 10) is supported under LTS for almost another year (June 30th, 2024), so it's getting security patches now, but its days are numbered. Even with the stripped down final images, developers still need to have a full environment to build everything, and that is going to get a lot harder when all the mirrors delete the packages for buster (like they did recently for stretch).

When following the manual installation instructions on the latest Raspberry Pi OS (based on Debian 11), the following error occurs when installing dependencies.

pi@raspberrypi:~ $ sudo apt update && sudo apt install -y wiringpi python3-pip \
   python3-numpy python-pil libjpeg-dev zlib1g-dev libopenjp2-7 \
   git python3-opencv python3-picamera libatlas-base-dev qrencode
Hit:1 http://security.debian.org/debian-security bullseye-security InRelease
Hit:2 http://deb.debian.org/debian bullseye InRelease
Hit:3 http://deb.debian.org/debian bullseye-updates InRelease
Hit:4 http://archive.raspberrypi.org/debian bullseye InRelease
Reading package lists... Done             
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Package wiringpi:armhf is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'wiringpi:armhf' has no installation candidate
E: Unable to locate package python-pil
kas-kaz commented 2 months ago

I was able to set up the development environment in a Raspberry Pi Zero 2 W using Raspios Bullseye Lite arm64.

The image I used was 2022-01-28-raspios-bullseye-arm64-lite.zip.

These are the steps I followed, adapted from the guide Raspberry Pi OS Build Instructions:

Initial setup

The only change from the original guide is making swap space permanent as it was useful for my setup.

sudo apt update

# Interface -> Activate camera and SPI
# Locale -> en_US.UTF-8
sudo raspi-config

# add `spidev.bufsiz=131072`
sudo nano /boot/cmdline.txt

# Change to `CONF_SWAPSIZE=1024`
sudo nano /etc/dphys-swapfile

sudo reboot

Install python3.10

The only difference is I didn't install package python-pil, but I installed the dependencies of that package in the following step.

Also I used make -j 3 to take advantage of the Quad Core (with -j 4 the use of Swap is so intensive that it is even slower).

The link to lsb_release was needed because the custom installation of Python 3.10 broke the package and the tests require it.

sudo apt install -y build-essential zlib1g-dev libncurses5-dev \
    libgdbm-dev libnss3-dev openssl libssl-dev libreadline-dev \
    libffi-dev wget libsqlite3-dev

# Grab the python3.10 source
wget https://www.python.org/ftp/python/3.10.10/Python-3.10.10.tgz
tar -xzvf Python-3.10.10.tgz
cd Python-3.10.10

# Compile
./configure --enable-optimizations
sudo make -j 3 altinstall

# Make python3.10 the default version
sudo update-alternatives --install /usr/bin/python python /usr/local/bin/python3.10 1
sudo update-alternatives --install /usr/bin/python3 python3 /usr/local/bin/python3.10 1

# Manually re-install `python3-apt`
sudo apt remove --purge python3-apt -y
sudo apt autoremove -y
sudo apt install python3-apt -y

# Link to lsb_release module, needed for tests
sudo ln -s /usr/share/pyshared/lsb_release.py /usr/local/lib/python3.10/site-packages/lsb_release.py

cd ..

Install dependencies

I had to download wiringpi-2.61-1-arm64 as the 64 bits version of the package is not available in apt. Version 2.61 (instead of 2.60 in the Buster setup) adds support for Zero 2 W.

The version of libzbar in apt in Bullseye is exactly the one required so no need to download the package.

# Install wiringpi with Zero 2 W support
wget https://github.com/WiringPi/WiringPi/releases/download/2.61-1/wiringpi-2.61-1-arm64.deb
sudo dpkg -i wiringpi-2.61-1-arm64.deb

sudo apt install -y python3-pip libjpeg-dev zlib1g-dev libopenjp2-7 git \
    python3-opencv python3-picamera libatlas-base-dev qrencode

# Install python-pil dependencies
sudo apt install -y libc6 libfreetype6 libimagequant0 libjpeg62-turbo \
    liblcms2-2 libtiff5 libwebp6 libwebpdemux2 libwebpmux3 zlib1g

sudo apt install -y libzbar0

Compile MMAL 64 bits libraries

This was the trickiest part. MMAL libraries used by PiCamera are not available for arm64. And everywhere I read about it they mention it is discouraged as it causes crashes. I tried capturing QR codes several times and nothing failed. We need to download userland project and make these modifications as explained here.

git clone https://github.com/raspberrypi/userland
cd userland
git config --global user.email "xxxxxx"
git config --global user.name "xxxxxx"
git revert f97b1af1b3e653f9da2c1a3643479bfd469e3b74
git revert e31da99739927e87707b2e1bc978e75653706b9c

# Add 'SET(CMAKE_SHARED_LINKER_FLAGS "-Wl,--no-as-needed")' before 
# line 'target_link_libraries(mmal mmal_core mmal_util mmal_vc_client vcos mmal_components)'
nano interface/mmal/CMakeLists.txt 

sudo apt install -y cmake

./buildme --aarch64
sudo cp /opt/vc/lib/*.so /usr/lib/aarch64-linux-gnu/

cd ..

Download the SeedSigner code:

git clone https://github.com/SeedSigner/seedsigner
cd seedsigner

Install Python pip dependencies:

I had to update RPi.GPIO to 0.7.1 as it added support for the newer version of GCC used in Bullseye.

python3 -m pip install -r requirements.txt
python3 -m pip install picamera==1.13 numpy==1.25.2 spidev==3.5 RPi.GPIO==0.7.1

The rest is the same! I was able to run the rests as was shown in the guide, and also could execute main.py and run the app correctly. I didn't check every single option though, just scanned some QRs, created a seed, signed a transaction, and restored the seed using QR in another SeedSigner device. All worked correctly.

I never contributed to this project and I didn't know if it was appropriate to create a PR with a new guide. If the contributors consider it is desirable I could do it of course.

hax0rbana-adam commented 2 months ago

Based on my experience in submitting pull requests to this project in the past, I expect your contribution would be very welcome. Debian 11 is still getting security patches from the Debian team, so it'd be good to have instructions on how to use it as a development environment, even if it does require jumping through all these hoops to get specific versions of Python, MMAL, and so forth.

I don't have a ton of free time, but I'd be happy to help by running through your updated guide once you have something for me to look at. I have various raspberry pi boards, and could test both 32- and 64-bit versions of the still supported versions of Debian (10-12).

I went on a bit of a side quest and tried to reproduce your efforts on the Debian 12 (32-bit) and document the issues there. It'd be nice to have the guide say things like "Make sure you have Python 3.10 or later. If you don't, you can compile it from source by doing this..." That way most of the steps can remain the same regardless of which version of Debian a person is using. Below are the places where I found issues with Debian 11 (arm64) vs 12 (armhf) in reproducing your work.

Python

The stable version of Debian (12 / bookworm) has Python 3.11, so we should already be good on this front.

Dependencies

GPIO

I don't see wiringpi in the apt repos, but downloading a .deb from GitHub worked fine. I'm also not sure if this is really needed since libgpiod ships with the bookworm image and that's also fast and written in C. But I digress.

python3-picamera

This doesn't appear to be in the stock repos of any version of Debian. https://packages.debian.org/search?searchon=names&keywords=python3-picamera

libtiff5 and libwebp6

Ah, this might be why you're not using the latest image of RaspiOS, libtiff5 was dropped from the repos. It looks like the same is true of libwebp6.

https://packages.debian.org/search?suite=default&section=all&arch=any&searchon=names&keywords=libtiff5

https://packages.debian.org/search?suite=default&section=all&arch=any&searchon=names&keywords=libwebp6

libzbar0

This one worked just fine.

raspberrypi/userland

I didn't try to compile MMAL since I couldn't get picamera.

Seedsigner repo

We have a ticket about this on Debian 12 https://github.com/SeedSigner/seedsigner/issues/430 and I put my comments on that ticket to try to keep things organized, but the punchline is that this step worked just fine.