search

rumpkernel / rumprun

The Rumprun unikernel and toolchain for various platforms
Other
1.14k stars 128 forks source link

Building for the RaspberryPi? #126

Open chrisruk opened 6 years ago

chrisruk commented 6 years ago

Hi,

I'm just wondering if anyone has managed to build rumprun for the Pi?

I'm trying the following so far:

docker run -it resin/rpi-raspbian:jessie

Then in the docker:

sudo apt-get update
sudo apt-get install make
sudo apt-get install git
sudo apt-get install gcc-4.9
sudo apt-get install zlib1g-dev

git clone https://github.com/dave-tucker/rumprun.git
git checkout raspberrypi

Modify: 

./src-netbsd/sys/arch/arm/include/aeabi.h
./src-netbsd/sys/sys/cdbr.h
./src-netbsd/lib/libc/locale/setlocale_local.h
./src-netbsd/sys/crypto/camellia/camellia.h
./src-netbsd/crypto/external/bsd/openssl/dist/crypto/camellia/camellia.h

adding:

typedef unsigned int size_t;

I also commented out

TARGETS+=   compiler_rt

In platform/Makefile.inc.

Then ran:

export HOST_CC=gcc-4.9
export CC=gcc-4.9

./build-rr.sh -b rpi -j4 hw -- -r -j4 -F ACFLAGS="-march=armv6k -march=armv6zk -mcpu=arm1176jz-s -mfloat-abi=hard"

Which gave:

>>
>> Finished ./build-rr.sh for hw
>>
>> For Rumprun developers (if you're not sure, you don't need it):
. "/root/tucker2/rumprun/./obj-evbearmhf-el-hw-raspberrypi/config"
>>
>> toolchain tuple: arm-rumprun-netbsdelf-eabihf
>> cc wrapper: arm-rumprun-netbsdelf-eabihf-gcc
>> installed to "/root/tucker2/rumprun/./rumprun-raspberrypi"
>>
>> ./build-rr.sh ran successfully

I then did:

Created file rumprun-bake2.conf:

#
# rumpbake default configuration file
#

version 20150930

#
# First, define internal convenience sets.
#
create _foundation "basic components for the Rumprun unikernel"
    add _foundation             \
        -lrumpvfs           \
        -lrumpkern_bmktc        \
        -lrumpkern_mman         \
        -lrumpdev           \
        -lrumpfs_tmpfs          \
        -lrumpnet_config        \
        -lrumpnet           \
        -lrumpdev_bpf           \
        -lrumpdev_vnd           \
        -lrumpdev_rnd

create _virtio "virtio drivers"
    add _virtio             \
        -lrumpdev_virtio_if_vioif   \
        -lrumpdev_virtio_ld     \
        -lrumpdev_virtio_viornd     \
        -lrumpdev_pci_virtio        \
        -lrumpdev_pci

create _audio "audio subsystem and some PCI audio device drivers"
    add _audio              \
        -lrumpdev_audio         \
        -lrumpdev_audio_ac97        \
        -lrumpdev_pci_auich     \
        -lrumpdev_pci_eap       \
        -lrumpdev_pci_hdaudio       \
        -lrumpdev_hdaudio_hdafg

create _netinet "TCP/IP (v4)"
    add _netinet                \
        -lrumpnet_netinet       \
        -lrumpnet_net           \
        -lrumpnet

create _netinet6 "TCP/IP (v6)"
    add _netinet6               \
        -lrumpnet_netinet6      \
        -lrumpnet_net           \
        -lrumpnet

create _netunix "local domain sockets"
    add _netunix                \
        -lrumpnet_local         \
        -lrumpnet

create _stdfs "selection of file system drivers"
    add _stdfs              \
        -lrumpfs_ffs            \
        -lrumpfs_cd9660         \
        -lrumpfs_ext2fs         \
        -lrumpdev_disk          \
    -lrumpvfs

create _pciether "PCI Ethernet NIC drivers"
    add _pciether               \
        -lrumpdev_pci_if_wm     \
        -lrumpdev_miiphy        \
        -lrumpdev_pci

create _usb "USB stack incl. PCI USB host controllers"
    add _usb                \
        -lrumpdev_pci_usbhc     \
        -lrumpdev_usb           \
        -lrumpdev_umass         \
        -lrumpdev_scsipi        \
        -lrumpdev_pci

# this is a config so that it can be assimilated
create _sysproxy "system call proxy support"
    add _sysproxy               \
        -lrumpkern_sysproxy

create _miconf "general useful MI/pseudo driver set"
    assimilate _miconf      \
        _foundation     \
        _stdfs          \
        _sysproxy

create hw_generic2 "generic targets, includes (almost) all drivers"
    assimilate hw_generic2      \
        _miconf         \

CC=rumprun/rumprun-raspberrypi/bin/arm-rumprun-netbsdelf-eabihf-gcc
$CC 1.c -o a.out
rumprun-bake -c rumprun-bake2.conf hw_generic2 test2.bin a.out 
qemu-system-arm -cpu arm1176 -m 128 -M raspi2 -kernel test2.bin
qemu-system-arm -nographic -cpu arm1176 -m 256 -M raspi2 -serial mon:stdio -kernel test2.bin

However no output is produced either graphically or via the serial output (I listed the command for the serial version above). And it seems to give high CPU usage on all cores.

Without the custom conf file I get:

root@56604f6e742f:~# rumprun-bake hw_generic a2.bin a.out 

!!!
!!! NOTE: rumprun-bake is experimental. syntax may change in the future
!!!

/usr/bin/ld: cannot find -lrumpdev_virtio_if_vioif
/usr/bin/ld: cannot find -lrumpdev_virtio_ld
/usr/bin/ld: cannot find -lrumpdev_virtio_viornd
/usr/bin/ld: cannot find -lrumpdev_pci_virtio
/usr/bin/ld: cannot find -lrumpdev_pci
/usr/bin/ld: cannot find -lrumpdev_audio_ac97
/usr/bin/ld: cannot find -lrumpdev_pci_auich
/usr/bin/ld: cannot find -lrumpdev_pci_eap
/usr/bin/ld: cannot find -lrumpdev_pci_hdaudio
/usr/bin/ld: cannot find -lrumpdev_hdaudio_hdafg
/usr/bin/ld: cannot find -lrumpdev_pci_if_wm
/usr/bin/ld: cannot find -lrumpdev_miiphy
/usr/bin/ld: cannot find -lrumpdev_pci_usbhc
collect2: error: ld returned 1 exit status

Any suggestions would be most appreciated!

Cheers Chris

zatkh commented 5 years ago

I am also wondering if there is any update on rumprun rpi or aarch64 support?

ananos commented 5 years ago

Hi @chrisruk, @zatkh,

we have a port of https://github.com/nabla-containers for aarch64. Maybe you'll find that useful:

https://github.com/cloudkernels/rumprun (branch solo5-tests-aarch64).

You can also find some info at https://blog.cloudkernels.net

cheers, Tassos

zatkh commented 5 years ago

That's great @ananos ,thanks, exactly was looking for aarch64 support of nabla to test with my solo5 backend extension! will try now

zatkh commented 5 years ago

Hi @ananos, I am testing this, it seems the blog post is not a match with current version of solo5 and nabla. (for example install solo5.o-->libsolo5_ukvm is before adding spt), so not sure what is solo5_stock and why you need two solo5 repos?

ananos commented 5 years ago

Hi @zatkh,

our aarch64 support has been accepted upstream so you should be able to build for a 64-bit RPi3 using the https://github.com/nabla-containers/rumprun

We are in the process of updating the blog posts to point to the updated repos. I'll ping you when we manage to do that.

In any case, please try using the Nabla version (we have tested hvt & spt on aarch64) and let us know how it goes!

ananos commented 5 years ago

Hi @zatkh,

we have updated the posts to point to the latest versions of the repos (nabla/solo5/cloudkernels). Please have a look and let us know if you manage to get something running, or if we missed something in the update process.

thanks!

zatkh commented 5 years ago

Hi @ananos ,

Sorry for the delay (I'm traveling), and thanks for letting me know. I quickly tested basic packages (hello,python3) on both my aarch64 server and rpi3 and didn't have time to fix some of the problems yet.

sqlite3: In functionsqlite3VXPrintf': /home/zahra/git/rumprun-packages/sqlite/build/sqlite3.c:22563: undefined reference to __lttf2' /home/zahra/git/rumprun-packages/sqlite/build/sqlite3.c:22582: undefined reference togttf2' /home/zahra/git/rumprun-packages/sqlite/build/sqlite3.c:22605: undefined reference to `getf2'`

compile rumpkern/atomic_add_16.o /home/pi/git/rumprun/src-netbsd/sys/rump/librump/rumpkern/../../../lib/libkern/../../../common/lib/libc/arch/arm/atomic/atomic_add_16.S:78: Error: selected processor does not support `strexh r2,r0,[ip]' in ARM mode

*** [atomic_add_16.o] Error code 1

nbmake[5]: stopped in /home/pi/git/rumprun/src-netbsd/sys/rump/librump/rumpkern 1 error

ananos commented 5 years ago

Hi @zatkh,

regarding the pi rumprun issue, I'm using the following and the build completes:

root@pi:/build/rumprun# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/aarch64-linux-gnu/6/lto-wrapper
Target: aarch64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 6.3.0-18+deb9u1' --with-bugurl=file:///usr/share/doc/gcc-6/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-6 --program-prefix=aarch64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libquadmath --enable-plugin --enable-default-pie --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-6-arm64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-6-arm64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-6-arm64 --with-arch-directory=aarch64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-multiarch --enable-fix-cortex-a53-843419 --enable-checking=release --build=aarch64-linux-gnu --host=aarch64-linux-gnu --target=aarch64-linux-gnu
Thread model: posix
gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1) 
root@pi:/build/rumprun# uname -a
Linux pi 4.16.18-v8+ #1 SMP PREEMPT Sat Sep 15 19:33:42 EEST 2018 aarch64 GNU/Linux
root@pi:/build/rumprun# dpkg -l |grep gcc   
ii  gcc                             4:6.3.0-4                      arm64        GNU C compiler
ii  gcc-6                           6.3.0-18+deb9u1                arm64        GNU C compiler
ii  gcc-6-base:arm64                6.3.0-18+deb9u1                arm64        GCC, the GNU Compiler Collection (base package)
ii  libgcc-6-dev:arm64              6.3.0-18+deb9u1                arm64        GCC support library (development files)
ii  libgcc1:arm64                   1:6.3.0-18+deb9u1              arm64        GCC support library

BTW on an arm server, the build completes using the following:

# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/aarch64-linux-gnu/5/lto-wrapper
Target: aarch64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.10' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libquadmath --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-arm64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-arm64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-arm64 --with-arch-directory=aarch64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-multiarch --enable-fix-cortex-a53-843419 --disable-werror --enable-checking=release --build=aarch64-linux-gnu --host=aarch64-linux-gnu --target=aarch64-linux-gnu
Thread model: posix
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.10) 
root@kmaxgw-arm64:~# uname -a
Linux arm64 4.17.0-00001-g010275f #2 SMP PREEMPT Sun Jul 22 12:33:44 EEST 2018 aarch64 aarch64 aarch64 GNU/Linux
root@kmaxgw-arm64:~# dpkg -l |grep gcc
ii  gcc-5                                  5.4.0-6ubuntu1~16.04.10                    arm64        GNU C compiler
ii  gcc-5-base:arm64                       5.4.0-6ubuntu1~16.04.10                    arm64        GCC, the GNU Compiler Collection (base package)
ii  libgcc-5-dev:arm64                     5.4.0-6ubuntu1~16.04.10                    arm64        GCC support library (development files)
ii  libgcc1:arm64                          1:8.1.0-12ubuntu1                          arm64        GCC support library

regarding the python3 issue, we are working on trying to understand why this happens. My initial assumption is that the long double type assumes it's 128-bit and the processor needs to use sw emulation for these built-in operations. If we manually link -lgcc from the system we're able to build python3 -- but we need to figure it out as this approach is far from clean ;-)

Will let you know as soon as we have something -- please do let us know if you manage to figure it out!

thanks!

QiZhang1997 commented 4 years ago

Hi @ananos When I tried to run unikernel on Raspberry pi 3B + following the instructions in your bloghere.

When I enter the following command/build/solo5/tenders/spt/solo5-spt --mem=32 --net=tap0 --disk=dummy ./bin/hello.spt.

I met an errorsolo5-spt: ./bin/hello.spt: No Solo5 ABI information found in executable

I don’t know what went wrong, do you have any suggestion? Thanks a lot, I'm looking forward to your reply. I'm very interested in your project.

ananos commented 4 years ago

just for reference, as the question was already answered offline -- rumprun over solo5 works with solo5 v0.4.1, so the solution to this question is to use tender binaries built from version v0.4.1

hazcod commented 4 years ago

I wonder if there is a way to run microkernels directly on the Pi without the use of Docker/firecracker/...