znailnetem / znail

Network emulator intended to run on a Raspberry Pi.
Apache License 2.0
16 stars 3 forks source link

Port to NanoPi R2S #3

Closed per-bohlin closed 3 years ago

per-bohlin commented 3 years ago

I think the NanoPi R2S could potentially be a good hardware platform for the znail:

https://www.friendlyarm.com/index.php?route=product/product&product_id=282

It has 2 GBit interfaces, TF-card slot for OS and a USB port that potentially could be used for a USB-storage device. The USB-storage could be used for saving TCP-dump, user settings that should be "portable" between devices and a text file with current IP-addresses of the interfaces written on boot to make it simpler to find on the network. It also supports a USB2LCD that can be used to display the IP-address.

Ubuntu core and FriendlyWRT are among the many OSes available.

DevConSoft is willing to sponsor this feature by paying for 1 set of the hardware (NanoPi R2S, case, MicroSD card, USB2LCD, shipping, VAT) up to 1000 SEK. Sponsorship require prior agreement between DevConSoft and the implementor and valid receipt of the purchase.

Original maintainers has first dibs on the sponsoring.

andni233 commented 3 years ago

Now that's a neat piece of hardware!

I especially like the USB2LCD module as that solves one of the major drawbacks of Znail on Raspberry PI, where discovering what IP Znail ended up getting assigned can be a bit tricky.

I think its a nice idea and I'd be happy to take a stab at porting Znail to this platform.

per-bohlin commented 3 years ago

@andni233 , I ordered a hardware kit from FriendlyElec (FriendlyArm) when I wrote the ticket. The package has arrived in Stockholm so I expect delivery in a couple of days. Would you like to get that kit or order one yourself and get reimbursed based on the receipt?

Risca commented 3 years ago

I've also ordered a kit :sweat_smile: but I don't need any reimbursement. We'll see if I even have time to work on it. From what I can tell, it's currently in Shanghai :smile:

andni233 commented 3 years ago

Ah FriendlyElec is warning about delays in delivery. I guess if I can take your kit that would be ideal as that would mean I likely have it for easter which is when I would have time to work on this.

per-bohlin commented 3 years ago

@andni233 , Even though the package has arrived in Stockholm, now the delivery company predicts delivery April 4th-14th, based on historical average handling times. I'll let you know when it has arrived and schedule a handover.

per-bohlin commented 3 years ago

@andni233, The package has finally arrived. I will reach out to you via SMS.

Risca commented 3 years ago

I've also received a NanoPi now =) So far I've only flashed FriendlyWrt on it and booted it up.

Not sure what the best approach would be here. Custom fork of FriendlyWrt or Ubuntu Core, or try and re-use some of the stuff used to generate the RPi image?

andni233 commented 3 years ago

There are plenty of options to choose from!

I've had a look at both Ubuntu Core and FriendlyWrt and while both could work they do have drawbacks.

Ubuntu Core just seems strange to me. I'm not sure how I feel about that it absolutely requires that you register your system with a Canonical account. The entire thing seems to me like Canonical wanting to lock you into their platform. I'm also not sure about the snap package format. It employs sandboxing and I'm not sure how well that would play with Znail. Persumably some major adaptations would be required.

FriendlyWrt looks like a nice alternative. However from what I can tell it is not that easy to create a custom image. It's buildroot based which I don't have much experience with. I don't know how difficult it would be to install Znails dependencies in that environment.

Another option which I currently think is looking promising is Armbian. Its Debian based so it is likely that it can be set up much the same as the Rasbperry Pi image. It supports a bunch of Arm based boards. The image creation process looks pretty straight forward and I'm sure we could hook into that to create images that come pre-package with Znail.

Risca commented 3 years ago

Buildroot is my specialty and build system of choice at home :wink: but it does have its benefits to use the same (or at least similar) build systems for both RPi and this. I suggest we investigate Armbian first 😁

Risca commented 3 years ago

Looks like NanoPi R2S is already supported by Armbian! :smiley_cat:

per-bohlin commented 3 years ago

@Risca , @andni233 , any progress?

Risca commented 3 years ago

Sorry, haven't had the time with all the vacation going on ^^ My vacation plans are slowing down now, so maybe I can find a slot this week =)

andni233 commented 3 years ago

I don't have anything complete, but I've made some progress and I have something that can build Armbian images. I'll clean this up a bit and see if I can make some more progress over the weekend. I'll push this to a branch here if you would like to collaborate :)

Risca commented 3 years ago

I'd love to! :D

Risca commented 3 years ago

I took a stab at armbian today and I think I have the needed config files ready, but I haven't tested a perfectly clean build nor have I integrated it with the znail repo.

Here are the config files I have: armbian-build/userpatches/lib.config:

PACKAGE_LIST_ADDITIONAL="$PACKAGE_LIST_ADDITIONAL bridge-utils ebtables iptables dnsmasq python3 python3-pip python3-venv python3-wheel git tcpdump vim"
PACKAGE_LIST_RM="$PACKAGE_LIST_RM smartmontools"

armbian-build/userpatches/config-znail.conf:

# Read build script documentation https://docs.armbian.com/Developer-Guide_Build-Options/
# for detailed explanation of these options and for additional options not listed here

KERNEL_ONLY="no"                        # leave empty to select each time, set to "yes" or "no" to skip dialog prompt
KERNEL_CONFIGURE="no"                   # leave empty to select each time, set to "yes" or "no" to skip dialog prompt
BUILD_MINIMAL="yes"
CLEAN_LEVEL="make,debs,oldcache"        # comma-separated list of clean targets: "make" = make clean for selected kernel and u-boot,
                                        # "debs" = delete packages in "./output/debs" for current branch and family,
                                        # "alldebs" = delete all packages in "./output/debs", "images" = delete "./output/images",
                                        # "cache" = delete "./output/cache", "sources" = delete "./sources"
                                        # "oldcache" = remove old cached rootfs except for the newest 8 files

REPOSITORY_INSTALL=""                   # comma-separated list of core modules which will be installed from repository
                                        # "u-boot", "kernel", "bsp", "armbian-config", "armbian-firmware"
                                        # leave empty to build from sources or use local cache

DEST_LANG="C.UTF-8"                     # sl_SI.UTF-8, en_US.UTF-8

# advanced
EXTERNAL_NEW="prebuilt"                 # compile and install or install prebuilt additional packages
INSTALL_HEADERS="no"                    # install kernel headers package
LIB_TAG="master"                        # change to "branchname" to use any branch currently available.
USE_TORRENT="yes"                       # use torrent network for faster toolchain and cache download
DOWNLOAD_MIRROR=""                      # set to "china" to use mirrors.tuna.tsinghua.edu.cn
CARD_DEVICE=""                          # device name /dev/sdx of your SD card to burn directly to the card when done

BRANCH="current"
RELEASE="buster"
BOARD="nanopi-r2s"
HOST="znail"

and armbian-build/userpatches/customize-image.sh:

#!/bin/bash

# arguments: $RELEASE $LINUXFAMILY $BOARD $BUILD_DESKTOP
#
# This is the image customization script

# NOTE: It is copied to /tmp directory inside the image
# and executed there inside chroot environment
# so don't reference any files that are not already installed

# NOTE: If you want to transfer files between chroot and host
# userpatches/overlay directory on host is bind-mounted to /tmp/overlay in chroot
# The sd card's root path is accessible via $SDCARD variable.

RELEASE=$1
LINUXFAMILY=$2
BOARD=$3
BUILD_DESKTOP=$4
OVERLAY="/tmp/overlay"

EnableKernelModules() {
    echo "sch_netem" >> /etc/modules
    echo "br_netfilter" >> /etc/modules
}

ConfigureNetwork() {
    install -v -m 700 "${OVERLAY}/update-network-interfaces" "/usr/local/bin"
    install -v -m 644 "${OVERLAY}/update-network-interfaces.service" "/etc/systemd/system/"
    rm -f "/etc/systemd/system/multi-user.target.wants/update-network-interfaces.service"
    ln -s "/etc/systemd/system/update-network-interfaces.service" "/etc/systemd/system/multi-user.target.wants/update-network-interfaces.service"
    # nanopi-r2s board support installs a udev rule to rename eth1 -> lan0.
    # We don't need that.
    rm -f "/etc/udev/rules.d/70-rename-lan.rules"
}

FixupHaveged() {
    # https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=866306
    sed -i -e 's/DAEMON_ARGS="\?\([^"]*\)"\?/DAEMON_ARGS="\1 -d 16"/g' "/etc/default/haveged"
}

InstallZnail() {
    install -d "/opt/znail"
    install -v -m 600 "${OVERLAY}/requirements.txt" "/opt/znail/requirements.txt"
    install -v -m 600 -D -t "/opt/znail" "${OVERLAY}"/*.whl
    pushd "/opt/znail"
    python3 -m venv venv
    source "venv/bin/activate"
    pip3 install wheel
    pip3 install -r requirements.txt
    pip3 install *.whl
    popd
    install -v -m 644 "${OVERLAY}/znail.service" "/etc/systemd/system/"
    rm -f "/etc/systemd/system/multi-user.target.wants/znail.service"
    ln -s "/etc/systemd/system/znail.service" "/etc/systemd/system/multi-user.target.wants/znail.service"
    # Fake hub-ctrl to make znail start
    rm -f "/usr/local/bin/hub-ctrl"
    ln -s "/usr/bin/true" "/usr/local/bin/hub-ctrl"
}

EnableKernelModules "$@"
ConfigureNetwork "$@"
FixupHaveged "$@"
InstallZnail "$@"

I'm positive that these can be improved upon, but it's a start. Still need to integrate it all into the znail repo as well.

Risca commented 3 years ago

I was able to control the power to the LAN port with some hacks. Needs to be cleaned up, though.

Basically, the LAN port of the NanoPi R2S is a USB device and there's a GPIO pin that controls the 5V power supply to that device. This gpio pin is by default claimed by the vcc_rtl8153 regulator, so no luck controlling the pin directly from userspace. When a regulator has no consumers (nobody which uses it), it will automatically turn off. That is, unless the always-on property is set, which is true for the NanoPi R2S.

The trick is to remove the always-on property from the device tree and use a reg-userspace-consumer device to control whether this regulator should be switched on or not. To do this, 3 things needs to be done:

See this page for some more info: https://www.studiofuga.com/2019/07/18/adding-a-user-space-power-switch-to-your-embedded-linux/

I didn't investigate if it would be possible to remove the vcc_rtl8153 regulator definition and just use the GPIO pin directly from userspace :shrug:

andni233 commented 3 years ago

I made a PR to integrate the suggested method for building the Armbian image with the Znail build system. I have verified that the resulting image can run on the NanoPi R2S. It works like a charm, and compared to the Raspberry Pi image creation its super quick!

I'll make a new release when this gets merged so we can have pre-built images available for both the NanoPi R2S and Raspbarry Pi available for download.

Risca commented 3 years ago

It took me maybe 2-3 hours to build the initial image, but subsequent image creations take 10-20 minutes. I wouldn't say super quick, but it depends on what you're comparing it with :wink:

Risca commented 3 years ago

@per-bohlin & @andni233, should we keep this issue open until support for disconnecting LAN cable is merged, or open a new issue?

per-bohlin commented 3 years ago

The disconnect feature can come in a separate issue or re-use this issue; I don't have a strong opinion on that. The publication of the new image for general download could be a good milestone for this issue and mark its closing - just a suggestion. If the disconnect feature comes later we can create a separate issue for it.

Risca commented 3 years ago

I agree, a new release should mark this issue as closed.

It looks like I can have a pull request for the disconnect feature up for review later today, so I suggest we hold off the release button for a few more days.

andni233 commented 3 years ago

I agree that a pre-built image with the disconnect feature would be a great milestone. Then all the features on the Raspberry Pi are also available on the NanoPi R2S.

I'll also start looking at getting the USB connected LCD display to show Znails current IP address.

Risca commented 3 years ago

I think it's time for a release! :smiley:

per-bohlin commented 3 years ago

Great work @andni233 and @Risca. Really cool that you could make it happen.

per-bohlin commented 3 years ago

If I was to ask for one more thing, it would be to update the "Getting Started" section to take the R2S into account. Perhaps a link to a trusted site on how to install the image.

andni233 commented 3 years ago

I think it's time for a release!

I had some issues with the build, but I think I got it working now. I'll publish some images soon!

Great work @andni233 and @Risca. Really cool that you could make it happen.

@Risca did all the difficult work, thank you @Risca!

If I was to ask for one more thing, it would be to update the "Getting Started" section to take the R2S into account. Perhaps a link to a trusted site on how to install the image.

Is something like #8 what you had in mind?

andni233 commented 3 years ago

New images are now available.

I realize the images could use some improved names. I manually renamed the Rasbian image to include "raspbian" in its name to clarify what is what. Next step would be to update the automation, and perhaps set up GitHub actions to do the release instead of from my PC.

per-bohlin commented 3 years ago

@andni233 , thanks for the "Getting Started" update. That was exactly what I had in mind.

per-bohlin commented 3 years ago

I think we can close this issue since the images for NanoPi R2S has been released.

Just for clarification, @Risca , did you manage to get the USB-poweroff feature to work? And is it included in the image? If not, could you write a ticket for it?

I'll give @Risca a couple of days to respone regarding the USB-poweroff, after which I will close this issue.

Update: @Risca , never mind - I found the implementation in the source code. Thanks.