Proposal for ARM64 support

ES-Six commented 2 years ago

Hello,

I'm writing this issue as it could help some users with arm64 devices (like RaspberryPi) to solve some weird errors like these when using OpenVPN option with this image :

Cannot ioctl TUNSETIFF tun: Function not implemented (errno=38)
ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2)

Solve Cannot ioctl TUNSETIFF tun: Function not implemented (errno=38)

I ran this docker image successfully on an aarch64 (ARM64) device without VPN enable using QEMU with Docker, it works fine without VPN. But when I enabled VPN, I got the error "Cannot ioctl TUNSETIFF tun: Function not implemented (errno=38)".

After searching and testing a lot, it appears that the problem was running this image with QEMU (QEMU allows a machine to run software that is not natively compatible with their CPU architecture, but this can affect compatibility with Linux devices like the /dev/tun devices).

If you encountered the error Cannot ioctl TUNSETIFF tun: Function not implemented (errno=38) you must use a native docker image without QEMU.

So as the problem is this docker image doesn't support ARM64 natively as there is no ARM64 build provided, I tried to build this image for ARM64 and I succeed with a tiny modification in the Dockerfile :

I changed the part to install ninja-build by this to use the native debian package that is compiled for ARM64 (as ninja-build GitHub release) doesn't provide a build of ninja-build for ARM64:

# Install Ninja
RUN apt update \
    && apt upgrade -y \
    && apt install -y --no-install-recommends \
    ca-certificates \
    curl \
    jq \
    unzip \
    ninja-build \
    && apt purge -y \
    ca-certificates \
    curl \
    jq \
    unzip \
    && apt-get clean \
    && apt --purge autoremove -y \
    && rm -rf \
    /var/lib/apt/lists/* \
    /tmp/* \
    /var/tmp/*

I changed the part to install cmake by this to use the ARM64 version :

# Install cmake
RUN apt update \
    && apt upgrade -y \
    && apt install -y  --no-install-recommends \
    ca-certificates \
    curl \
    jq \
    && CMAKE_ASSETS=$(curl -sX GET "https://api.github.com/repos/Kitware/CMake/releases" | jq '.[] | select(.prerelease==false) | .assets_url' | head -n 1 | tr -d '"') \
    && CMAKE_DOWNLOAD_URL=$(curl -sX GET ${CMAKE_ASSETS} | jq '.[] | select(.name | match("Linux-aarch64.sh";"i")) .browser_download_url' | tr -d '"') \
    && curl -o /opt/cmake.sh -L ${CMAKE_DOWNLOAD_URL} \
    && chmod +x /opt/cmake.sh \
    && /bin/bash /opt/cmake.sh --skip-license --prefix=/usr \
    && rm -rf /opt/* \
    && apt purge -y \
    ca-certificates \
    curl \
    jq \
    && apt-get clean \
    && apt --purge autoremove -y \
    && rm -rf \
    /var/lib/apt/lists/* \
    /tmp/* \
    /var/tmp/*

Then I built the image :

docker build --no-cache -t custombuiltimage/qbittorent .

Then run this image on raspberry pi :

docker run  -d \
              -v /media/hdd/config:/config \
              -v /media/hdd/downloads:/downloads \
              -e "VPN_ENABLED=yes" \
              -e "VPN_TYPE=[YOUR CONFIG]" \
              -e "LAN_NETWORK=[YOUR CONFIG]" \
              -e "LEGACY_IPTABLES=yes" \
              -e "ENABLE_SSL=yes" \
              -e "NAME_SERVERS=[YOUR CONFIG]" \
              -e "HEALTH_CHECK_HOST=[YOUR CONFIG]" \
              -e "HEALTH_CHECK_SILENT=0" \
              -e "HEALTH_CHECK_INTERVAL=3600" \
              -e "RESTART_CONTAINER=no" \
              -e "PUID=[YOUR CONFIG]" \
              -e "PGID=[YOUR CONFIG]" \
              --device /dev/net/tun \
              -p 8080:8080 \
              --cap-add NET_ADMIN \
              --sysctl "net.ipv4.conf.all.src_valid_mark=1" \
              --sysctl "net.ipv6.conf.all.disable_ipv6=0" \
              --restart unless-stopped \
              --privileged \
              custombuiltimage/qbittorent

And now with this native ARM64 build of the DyonR/docker-qbittorrentvpn image it works fine.

Solve cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2)

Note : I also added the extra parameter --device /dev/net/tun to my docker run command to allow the image to access the /dev/net/tun device as I was having the error because it does not seem all docker installations can access this device natively so this extra parameter allows docker to do it.

The error solved by adding --device /dev/net/tun was :

ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2)

Proposal ARM64 support

I also create this issue as a proposal to suggest the publication of a multiarch image to dockerhub with ARM64 support for this image (it's possible to create build for ARM64 devices without having one using docker buildx that allows creating ARM64 builds with a x86_64 compatible machine.

I tested this image and I find it very useful, it automates complicated stuff like VPN in a docker with killswitch iptable, that's why I did some research to make this image compatible with ARM64 devices (ex : devices like raspberry pi, etc.). I also like that qbittorent, libtorrent-rasterbar and boost are compiled by the image.

ES-Six commented 2 years ago

Here is the full modified dockerfile : Dockerfile.zip

For now, to use the DyonR/docker-qbittorrentvpn docker image one ARM64 devices with a VPN enabled you need to :

Clone the master branch of the DyonR/docker-qbittorrentvpn repository (this repository)
Then replace Dockerfile by the custom docker file given above.
Then you will be able to build this image for ARM64 devices : docker build --no-cache -t custombuiltimage/docker-qbittorrentvpn .

SoloW1ngPixy commented 2 years ago

Hi,

Thanks for the solution. Tried on a RPI2 but process get killed with Error 137 during compilation. Seem that RPI2 is too old for this task (not enough memory error). It is possible to optimize that ?


common.copy /usr/lib/cmake/boost_fiber-1.80.0/libboost_fiber-variant-shared.cmake  
boost-install.generate-cmake-variant- bin.v2/libs/filesystem/build/gcc-10/release/threading-multi/visibility-hidden/libboost_filesystem-variant-shared.cmake  
common.copy /usr/lib/cmake/boost_filesystem-1.80.0/libboost_filesystem-variant-shared.cmake  
Killed  
The command '/bin/sh -c apt update     && apt upgrade -y      && apt install -y --no-install-recommends     curl     ca-certificates     g++     libxml2-utils     && BOOST_VERSION_DOT=$(curl -sX GET "https://www.boost.org/feed/news.rss" | xmllint --xpath '//rss/channel/item/title/text()' - | awk -F 'Version' '{print $2 FS}' - | sed -e 's/Version//g;s/\ //g' | xargs | awk 'NR==1{print $1}' -)     && BOOST_VERSION=$(echo ${BOOST_VERSION_DOT} | head -n 1 | sed -e 's/\./_/g')     && curl -o /opt/boost_${BOOST_VERSION}.tar.gz -L https://boostorg.jfrog.io/artifactory/main/release/${BOOST_VERSION_DOT}/source/boost_${BOOST_VERSION}.tar.gz     && tar -xzf /opt/boost_${BOOST_VERSION}.tar.gz -C /opt     && cd /opt/boost_${BOOST_VERSION}     && ./bootstrap.sh --prefix=/usr     && ./b2 --prefix=/usr install     && cd /opt     && rm -rf /opt/*     && apt -y purge     curl     ca-certificates     g++     libxml2-utils     && apt-get clean     && apt --purge autoremove -y     && rm -rf     /var/lib/apt/lists/*     /tmp/*     /var/tmp/*' returned a non-zero code: 137`

ES-Six commented 2 years ago

Hi,

Thanks for the solution. Tried on a RPI2 but process get killed with Error 137 during compilation. Seem that RPI2 is too old for this task (not enough memory error). It is possible to optimize that ?

common.copy /usr/lib/cmake/boost_fiber-1.80.0/libboost_fiber-variant-shared.cmake  
boost-install.generate-cmake-variant- bin.v2/libs/filesystem/build/gcc-10/release/threading-multi/visibility-hidden/libboost_filesystem-variant-shared.cmake  
common.copy /usr/lib/cmake/boost_filesystem-1.80.0/libboost_filesystem-variant-shared.cmake  
Killed  
The command '/bin/sh -c apt update     && apt upgrade -y      && apt install -y --no-install-recommends     curl     ca-certificates     g++     libxml2-utils     && BOOST_VERSION_DOT=$(curl -sX GET "https://www.boost.org/feed/news.rss" | xmllint --xpath '//rss/channel/item/title/text()' - | awk -F 'Version' '{print $2 FS}' - | sed -e 's/Version//g;s/\ //g' | xargs | awk 'NR==1{print $1}' -)     && BOOST_VERSION=$(echo ${BOOST_VERSION_DOT} | head -n 1 | sed -e 's/\./_/g')     && curl -o /opt/boost_${BOOST_VERSION}.tar.gz -L https://boostorg.jfrog.io/artifactory/main/release/${BOOST_VERSION_DOT}/source/boost_${BOOST_VERSION}.tar.gz     && tar -xzf /opt/boost_${BOOST_VERSION}.tar.gz -C /opt     && cd /opt/boost_${BOOST_VERSION}     && ./bootstrap.sh --prefix=/usr     && ./b2 --prefix=/usr install     && cd /opt     && rm -rf /opt/*     && apt -y purge     curl     ca-certificates     g++     libxml2-utils     && apt-get clean     && apt --purge autoremove -y     && rm -rf     /var/lib/apt/lists/*     /tmp/*     /var/tmp/*' returned a non-zero code: 137`

Personally, I did this on RPI 4. I don't know if it will work with ARMHF instead of ARM64. RPI 2 is not an ARM64 device, you may need to adapt the Dockerfile a bit more.

There is no way to optimize the RAM consumption when compiling boost (it's a big library) .

But there is another solution : Use docker BuildX to compile this image for ARMHF from another device with more RAM (you can do this from any standard AMD/Intel 64 bit computers). Docker BuildX allows you to build images for other CPU architectures like ARMHF or ARM64 (and also it allows to build multi aarch docker images).

Then import the compiled image (for ARMHF, it's important) to the RPI 2.

From there, if the image doesn't start because of RAM is too low, try with a swap (at least 512MB).

cad0p commented 1 year ago

Hi @ES-Six, thanks for your work!

I have a Raspberry Pi 4, but trying to build the image resulted in this error:

[95/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/socket_type.cpp.o
[96/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/storage_piece_set.cpp.o
[97/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/string_util.cpp.o
[98/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/time.cpp.o
[99/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/timestamp_history.cpp.o
[100/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/storage_utils.cpp.o
[101/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/storage.cpp.o
[102/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/session_impl.cpp.o
FAILED: CMakeFiles/torrent-rasterbar.dir/src/session_impl.cpp.o 
/usr/bin/c++ -DBOOST_ASIO_ENABLE_CANCELIO -DBOOST_ASIO_HAS_STD_CHRONO -DBOOST_EXCEPTION_DISABLE -DTORRENT_BUILDING_LIBRARY -DTORRENT_BUILDING_SHARED -DTORRENT_USE_ICONV -DTORRENT_USE_LIBCRYPTO -DTORRENT_USE_OPENSSL -D_FILE_OFFSET_BITS=64 -Dtorrent_rasterbar_EXPORTS -I/opt/libtorrent-rasterbar-1.2.18/include -O3 -DNDEBUG -fPIC -fvisibility=hidden -fvisibility-inlines-hidden -Wall -Wextra -Wpedantic -Wvla -Wno-format-zero-length -ftemplate-depth=512 -fexceptions -pthread -std=gnu++17 -MD -MT CMakeFiles/torrent-rasterbar.dir/src/session_impl.cpp.o -MF CMakeFiles/torrent-rasterbar.dir/src/session_impl.cpp.o.d -o CMakeFiles/torrent-rasterbar.dir/src/session_impl.cpp.o -c /opt/libtorrent-rasterbar-1.2.18/src/session_impl.cpp
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
[103/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/torrent_info.cpp.o
[104/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/torrent_handle.cpp.o
[105/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/torrent.cpp.o
ninja: build stopped: subcommand failed.
The command '/bin/sh -c apt update     && apt upgrade -y     && apt install -y --no-install-recommends     build-essential     ca-certificates     curl     jq     libssl-dev     && LIBTORRENT_ASSETS=$(curl -sX GET "https://api.github.com/repos/arvidn/libtorrent/releases" | jq '.[] | select(.prerelease==false) | select(.target_commitish=="RC_1_2") | .assets_url' | head -n 1 | tr -d '"')     && LIBTORRENT_DOWNLOAD_URL=$(curl -sX GET ${LIBTORRENT_ASSETS} | jq '.[0] .browser_download_url' | tr -d '"')     && LIBTORRENT_NAME=$(curl -sX GET ${LIBTORRENT_ASSETS} | jq '.[0] .name' | tr -d '"')     && curl -o /opt/${LIBTORRENT_NAME} -L ${LIBTORRENT_DOWNLOAD_URL}     && tar -xzf /opt/${LIBTORRENT_NAME}     && rm /opt/${LIBTORRENT_NAME}     && cd /opt/libtorrent-rasterbar*     && cmake -G Ninja -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/usr -DCMAKE_CXX_STANDARD=17     && cmake --build build --parallel $(nproc)     && cmake --install build     && cd /opt     && rm -rf /opt/*     && apt purge -y     build-essential     ca-certificates     curl     jq     libssl-dev     && apt-get clean     && apt --purge autoremove -y      && rm -rf     /var/lib/apt/lists/*     /tmp/*     /var/tmp/*' returned a non-zero code: 1

ES-Six commented 1 year ago

Hi @ES-Six, thanks for your work!

I have a Raspberry Pi 4, but trying to build the image resulted in this error:

[95/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/socket_type.cpp.o
[96/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/storage_piece_set.cpp.o
[97/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/string_util.cpp.o
[98/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/time.cpp.o
[99/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/timestamp_history.cpp.o
[100/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/storage_utils.cpp.o
[101/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/storage.cpp.o
[102/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/session_impl.cpp.o
FAILED: CMakeFiles/torrent-rasterbar.dir/src/session_impl.cpp.o 
/usr/bin/c++ -DBOOST_ASIO_ENABLE_CANCELIO -DBOOST_ASIO_HAS_STD_CHRONO -DBOOST_EXCEPTION_DISABLE -DTORRENT_BUILDING_LIBRARY -DTORRENT_BUILDING_SHARED -DTORRENT_USE_ICONV -DTORRENT_USE_LIBCRYPTO -DTORRENT_USE_OPENSSL -D_FILE_OFFSET_BITS=64 -Dtorrent_rasterbar_EXPORTS -I/opt/libtorrent-rasterbar-1.2.18/include -O3 -DNDEBUG -fPIC -fvisibility=hidden -fvisibility-inlines-hidden -Wall -Wextra -Wpedantic -Wvla -Wno-format-zero-length -ftemplate-depth=512 -fexceptions -pthread -std=gnu++17 -MD -MT CMakeFiles/torrent-rasterbar.dir/src/session_impl.cpp.o -MF CMakeFiles/torrent-rasterbar.dir/src/session_impl.cpp.o.d -o CMakeFiles/torrent-rasterbar.dir/src/session_impl.cpp.o -c /opt/libtorrent-rasterbar-1.2.18/src/session_impl.cpp
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
[103/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/torrent_info.cpp.o
[104/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/torrent_handle.cpp.o
[105/156] Building CXX object CMakeFiles/torrent-rasterbar.dir/src/torrent.cpp.o
ninja: build stopped: subcommand failed.
The command '/bin/sh -c apt update     && apt upgrade -y     && apt install -y --no-install-recommends     build-essential     ca-certificates     curl     jq     libssl-dev     && LIBTORRENT_ASSETS=$(curl -sX GET "https://api.github.com/repos/arvidn/libtorrent/releases" | jq '.[] | select(.prerelease==false) | select(.target_commitish=="RC_1_2") | .assets_url' | head -n 1 | tr -d '"')     && LIBTORRENT_DOWNLOAD_URL=$(curl -sX GET ${LIBTORRENT_ASSETS} | jq '.[0] .browser_download_url' | tr -d '"')     && LIBTORRENT_NAME=$(curl -sX GET ${LIBTORRENT_ASSETS} | jq '.[0] .name' | tr -d '"')     && curl -o /opt/${LIBTORRENT_NAME} -L ${LIBTORRENT_DOWNLOAD_URL}     && tar -xzf /opt/${LIBTORRENT_NAME}     && rm /opt/${LIBTORRENT_NAME}     && cd /opt/libtorrent-rasterbar*     && cmake -G Ninja -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/usr -DCMAKE_CXX_STANDARD=17     && cmake --build build --parallel $(nproc)     && cmake --install build     && cd /opt     && rm -rf /opt/*     && apt purge -y     build-essential     ca-certificates     curl     jq     libssl-dev     && apt-get clean     && apt --purge autoremove -y      && rm -rf     /var/lib/apt/lists/*     /tmp/*     /var/tmp/*' returned a non-zero code: 1

Hi,

It's a RAM issue, your device doesn't have enough free RAM to compile the files.

fatal error: Killed signal terminated program cc1plus this error means the compiler has been interrupted unexpectedly by the system.

Try increase your SWAP and free some RAM by stopping temporarly some processes (ex : try to stop some docker containers temporarly)

AYastrebov commented 1 year ago

@ES-Six thanks for instructions! Successfully built for orangePi 4 on dietPi

DyonR / docker-qbittorrentvpn