search

lkl / linux

Linux kernel source tree
https://lkl.github.io/
Other
807 stars 136 forks source link

readme

Introduction

LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code as extensively as possible with minimal effort and reduced maintenance overhead.

Examples of how LKL can be used are: creating userspace applications (running on Linux and other operating systems) that can read or write Linux filesystems or can use the Linux networking stack, creating kernel drivers for other operating systems that can read Linux filesystems, bootloaders support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be directly linked by applications. The API offered by LKL is based on the Linux system call interface.

LKL is implemented as an architecture port in arch/lkl. It uses host operations defined by the application or a host library (tools/lkl/lib).

Supported hosts

The supported hosts for now are POSIX and Windows userspace applications.

Building LKL, the host library and LKL based tools

$ make -C tools/lkl

will build LKL as a object file, it will install it in tools/lkl/lib together with the headers files in tools/lkl/include then will build the host library, tests and a few of application examples:

Building LKL on FreeBSD

$ pkg install binutils gcc gnubc gmake gsed coreutils bison flex python argp-standalone

#Prefer ports binutils and GNU bc(1):
$ export PATH=/sbin:/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/lib64/ccache

$ gmake -C tools/lkl

Building LKL on Ubuntu

$ sudo apt-get install libfuse-dev libarchive-dev xfsprogs

# Optional, if you would like to be able to run tests
$ sudo apt-get install btrfs-tools
$ pip install yamlish junit_xml

$ make -C tools/lkl

# To check that everything works:
$ cd tools/lkl
$ make run-tests

Building LKL for Windows

In order to build LKL for Win32 the mingw cross compiler needs to be installed on the host (e.g. on Ubuntu the following packages are required: binutils-mingw-w64-i686, gcc-mingw-w64-base, gcc-mingw-w64-i686 mingw-w64-common, mingw-w64-i686-dev).

Due to a bug in mingw regarding weak symbols the following patches needs to be applied to mingw-binutils:

https://sourceware.org/ml/binutils/2015-10/msg00234.html

and i686-w64-mingw32-gas, i686-w64-mingw32-ld and i686-w64-mingw32-objcopy need to be recompiled.

With that pre-requisites fullfilled you can now build LKL for Win32 with the following command:

$ make CROSS_COMPILE=i686-w64-mingw32- -C tools/lkl

Building LKL on Windows

To build on Windows, certain GNU tools need to be installed. These tools can come from several different projects, such as cygwin, unxutils, gnu-win32 or busybox-w32. Below is one minimal/modular set-up based on msys2.

Common build dependencies:

General considerations:

For MSYS2 (and Cygwin):

Msys2 will install a gcc tool chain as part of the base-devel bundle. Binutils (2.26) is already patched for NT weak externals. Using the msys2 shell, cd to the lkl sources and run:

$ make -C tools/lkl

For MinGW:

Install mingw-w64-i686-toolchain via pacman, mingw-w64-i686-binutils (2.26) is already patched for NT weak externals. Start a MinGW Win32 shell (64-bit will not work, see above) and run:

$ make -C tools/lkl

LKL hijack library

LKL hijack library (liblkl-hijack.so) is used to replace system calls used by an application on the fly so that the application can use LKL instead of the kernel of host operating system. LD_PRELOAD is used to dynamically override system calls with this library when you execute a program.

You can usually use this library via a wrapper script.

$ cd tools/lkl
$ ./bin/lkl-hijack.sh ip address show

In order to configure the behavior of LKL, a json file can be used. You can specify json file with environmental variables (LKL_HIJACK_CONFIG_FILE). If there is nothing specified, LKL tries to find with the name 'lkl-hijack.json' for the configuration file. You can also use the old-style configuration with environmental variables (e.g., LKL_HIJACK_NET_IFTYPE) but those are overridden if a json file is specified.

     $ cat conf.json
     {
       "gateway":"192.168.0.1",
       "gateway6":"2001:db8:0:f101::1",
       "debug":"1",
       "singlecpu":"1",
       "sysctl":"net.ipv4.tcp_wmem=4096 87380 2147483647",
       "boot_cmdline":"ip=dhcp",
       "interfaces":[
               {
                       "mac":"12:34:56:78:9a:bc",
                       "type":"tap",
                       "param":"tap7",
                       "ip":"192.168.0.2",
                       "masklen":"24",
                       "ifgateway":"192.168.0.1",
                       "ipv6":"2001:db8:0:f101::2",
                       "masklen6":"64",
                       "ifgateway6":"2001:db8:0:f101::1",
                       "offload":"0xc803"
               },
               {
                       "mac":"12:34:56:78:9a:bd",
                       "type":"tap",
                       "param":"tap77",
                       "ip":"192.168.1.2",
                       "masklen":"24",
                       "ifgateway":"192.168.1.1",
                       "ipv6":"2001:db8:0:f102::2",
                       "masklen6":"64",
                       "ifgateway6":"2001:db8:0:f102::1",
                       "offload":"0xc803"
               }
       ]
     }
     $ LKL_HIJACK_CONFIG_FILE="conf.json" lkl-hijack.sh ip addr s

The following are the list of keys to describe a JSON file.

     "mount": "proc,sysfs"

LKL hijack library with zpoline

zpoline is an alternative to syscall hijack based on LD_PRELOAD, which is still default on LKL. The zpoline library works with binary rewrites to the loaded programs upon instantiation, then load hook function for the original syscalls. The LKL hijack library works together with zpoline by loading LKL.

zpoline currently only works on x86_64 machines.

To use the zpoline-enabled hijack library, please follow the instruction below.

Suppose zpoline is downloaded at ../zpoline and already build before LKL build.

zpoline rewrites the memory address 0x0 to hook syscalls, but non-root users don't have a privilege to operate that address. The following configuration allows us to use zpoline without root privilege.

sudo sh -c "echo 0 > /proc/sys/vm/mmap_min_addr"

then, execute command with the environment variable LKL_HIJACK_ZPOLINE=1.

LKL_HIJACK_ZPOLINE=1 LKL_HIJACK_CONFIG_FILE=lkl-tap.json \
  ./tools/lkl/bin/lkl-hijack.sh ping www.google.com

The file lkl-tap.json can be prepared like this.

{
    "gateway": "172.17.0.1",
    "nameserver": "8.8.8.8",
    "interfaces": [
        {
            "ip": "172.17.0.39",
            "masklen": "16",
            "mac": "00:0d:0b:94:4e:97",
            "param": "tap0",
            "type": "tap"
        }
    ],
}

With the preload hijack library, which is the default one, it uses the host name resolver and if the host uses a nameserver, defined at /etc/resolv.conf, like 127.0.0.53, is not accepting DNS requests, in a view of the LKL instance.

But with zpoline, it can successfully replace all syscalls for name resolution so can ping with a name.

FAQ

Q: How is LKL different from UML?

A: UML prodivides a full OS environment (e.g. user/kernel separation, user processes) and also has requirements (a filesystem, processes, etc.) that makes it hard to use it for standalone applications. UML also relies heavily on Linux hosts. On the other hand LKL is designed to be linked directly with the application and hence does not have user/kernel separation which makes it easier to use it in standalone applications.

Q: How is LKL different from LibOS?

A: LibOS re-implements high-level kernel APIs for timers, softirqs, scheduling, sysctl, SLAB/SLUB, etc. LKL behaves like any arch port, implementing the arch level operations requested by the Linux kernel. LKL also offers a host interface so that support for multiple hosts can be implemented.