Turmvilla scenario

[nfeske]

At this year's Hack'n'Hike, I kicked off my transition of my day-to-day computing environment from Linux to Genode. This issue will document the progress.

The transition will take several steps. The first step is a minimal Genode environment that allows the hosting of a rich Linux environment inside a virtual machine. This rich Linux environment basically contains my current Linux-based setup with the sole difference of running it inside a VM. Once this setup works flawlessly, I will move my individual computing tasks to the Genode world.

I picked a Lenovo x201 as the hardware for this undertaking because those machines are very cheaply available (refurbished), they come with AMT support, and the hardware is known to be well supported by Genode's device drivers.

How does such a minimal Genode system look like? It needs to satisfy at least the following functional requirements:

• A VM capable of executing a commodity Linux distro (I am used to Debian-based distributions. So here is my preference). VirtualBox fits the bill here.
• A disk driver that allows VirtualBox to access its VDI image and my encrypted home partition, and that allows Genode to access a dedicated Genode partition. Our AHCI driver will do that.
• A way to use different partitions of my hard disk:
  • The first partition will be called "Genode partition". It contains an ext2 file system with the Genode binaries and other data that will be accessed by the Genode system. This partition can be accessed by the Grub2 boot loader to load the Genode scenario. The Genode partition will contain the VDI image of the rich Linux system.
  • The second partition will be my encrypted home partition that I will assign directly to the Linux VM. Initially, the block encryption will be performed by the Linux kernel running in the VM. Of course, we don't gain any security over the traditional use of Linux here. But the setup should be considered as an intermediate one. We leave moving the block encryption into a dedicated Genode component for later.
  • The third partition will contain a minimalistic fallback Linux system that is able to access the Genode partition and run the rich Linux inside VirtualBox (but running on Linux). This fallback Linux can be used to fix things in case we damage the Genode system.
• A way to interactively start and stop the Linux VM. This part will be played by CLI monitor.
• A way to transfer data back and forth between the rich Linux world and the Genode world. For this task, we will use a Noux environment combined with VirtualBox' shared-folder support.
• A way to manipulate/configure the Genode system. This is the designated role of the Noux environment.
• Wireless networking. Here, our wifi_drv (our port of the Intel wireless stack) comes into play.
• A GUI that allows me to interact with the rich Linux system and the Genode components. It goes without saying that I will use our window manager for that.

As a matter of fact, I am typing this text on exactly this setup (Firefox running on Xubuntu, running in VirtualBox, running on Genode, running on NOVA).

[nfeske]

For setting up the basic scenario, I followed the excellent step-by-step instructions as compiled by @cproc: https://github.com/cproc/genode/blob/vbox_vm_linux/doc/virtualbox_setup.txt

I am using 32-bit Xubuntu 14.04 as my rich Linux system.

The Genode branch with the Turmvilla scenario can be found at https://github.com/nfeske/genode/commits/turmvilla Note that this branch is a topic branch. So I may rebase it from time to time. So please don't rely on it but rather create your own branch and cherry pick the commits you want.

The current version present a CLI monitor window immediately after booting. The CLI monitor can be used to start the rich Linux using the start linux command. The Noux environment can be started via start noux. The Noux environment has the following features:

• It contains coreutils, vim, and bash.
• The Genode partition is mounted at /genode.
• The RAM file system shared with VirtualBox is mounted at /shared.
• The LOG output of all CLI monitor children as well as some low-level services (like wifi_drv) is available at /shared/log.
• The config file system (containing only the wifi_drv config for now) is mounted at /config.
• A few interesting reports are available at /reports. The the time being, the most interesting one is the wlan_accesspoints report.
• The wifi_drv can be configured interactively by editing /config/wlan_configuration via vim. When saving the file, the wifi_drv will automatically pick up the new configuration.

Once wifi_drv is connected, the rich Linux running in VirtualBox is able to access the network.

[nfeske]

I updated my topic branch https://github.com/nfeske/genode/commits/turmvilla today. It is rebased on the current genodelabs/master branch. Hence, it uses the new acpi/pci infrastructure now.

Additionally, the new version comes with the following improvements

• Window manager
  • Added support for focus-follows-mouse (formerly, the user had to click into a window to set the keyboard focus)
  • Added support for click-to-raise (formerly, the user had to click on the window title to raise the window)
• The use of proper pointer shapes when the mouse is positioned over the rich-Linux window. The pointer is actually the nitpicker pointer. Although the shape is defined by the Guest OS, the response to motion events is super-smooth now.
• Improved interactive performance, e.g., by prioritizing real-time-critical components (like nitpicker, the window manager, input drivers) accordingly.
• Reports mounted in the Noux VFS are updated after the initialization. Hence, /reports/wlan_state and /reports/wlan_accesspoints contain always the most current state of the wifi driver.
• The nitpicker configuration can be updated on-the-fly by editing /config/nitpicker.config using Vim in Noux.
• The subsystems visible at the cli_monitor can be edited and extended dynamically by adding or editing the files at /genode/subsystems/. E.g. before starting the "linux" subsystem, the window size of the guest window can be adjusted by editing the configuration of the nit_fb instance in the linux.subsystem file.

To attain a reasonably good interactive performance, I am using the NOVA patch posted in issue #1566. Note, however, that this patch is only expected to work on Lenovo x201 machines. A real solution is still pending.

I am using the Turmvilla scenario exclusively since two weeks now. My original Linux machine is just sitting idle and logs the low-level serial output of the x201 - just for the case something goes wrong. The performance within the rich Linux environment suffices for my daily computing tasks (email, browsing, light development, text editing) but it still holds me back from tackling complex development tasks, in particular building components using Qt5. I hope that #1553 will cure that.

[nfeske]

The commit https://github.com/nfeske/genode/commit/c3b143317c95af1940129185d63b403d9dbd8308 significantly improves the performance of the Linux guest while accessing the home partition. I wonder why the VFS block file system uses a default block-buffer count of 1?

[chelmuth]

Good catch, since other test cases don't increase the block count either. Unfortunately, the block-session gets constructed with a packet-stream buffer of 128 KiB only.

[nfeske]

I think it would make much sense to make the block-session argument configurable and derive the value for the block-buffer count from that.

[chelmuth]

You're absolutely right.

[nfeske]

I rebased my turmvilla branch to genodelabs/master. I.e., it has been adapted to the reworked platform driver.

[nfeske]

Today, I rebased the turmvilla branch to Genode 15.08. Apart from the use of the latest version of all components, the most significant additions are

• A customization of the vm_linux.vbox configuration to enforce a finer granularity of the VirtualBox timer manager (TM). The use of 5 milliseconds instead of 10 visibly improves the responsiveness of the guest.
• Enabling SMP support for the Linux guest. I have set the number of CPUs to 4 (vm_linux.vbox) according to the used hardware (Lenovo x201). Thanks @cproc for improving me life so much! Also, the load display has become much more entertaining. :-)
• NOVA's new kernel-resource management works as intended. In situations where I could reliably produce an out-of-memory condition in the old version (using Noux, by executing /bin/true in an infinite loop), the new version does not trigger a kernel panic any more. Thanks @alex-ab for solving this long-standing issue!

[chelmuth]

:+1:

[nfeske]

I just rebased my turmvilla branch to the current version master. The most visible changes are:

• The changed x-ray mode handling, which has been moved from nitpicker to external components. This allows us to tailor the x-ray activation to all kinds of policies. In the scenario, x-ray is activated by hovering the window decorations, or toggled via F12. (#1690)
• The rules for configuring nitpicker and the decorator depending on the x-ray mode can be adjusted live via noux by editing the files config/nitpicker.config and config/decorator.config.
• The window decorations feature a closer and maximizer. The controls are not always functional. But the maximizer already works for the VirtualBox window.

[nfeske]

I rebased my turmvilla branch (https://github.com/nfeske/genode/commits/turmvilla) to Genode 15.11 just now. The following changes are worth noting:

• By default, the new themed decorator is used. The decorator config (i.e., the colors of the decorations) can be tweaked at runtime by editing config/decorator.config in noux.
• The decorator can be replaced at runtime by editing config/decorator_init.config. Just replace the binary name "themed_decorator" with "decorator" to activate the original (Motif-inspired) decorator.
• The new Intel framebuffer driver is used by default now. For using the VESA driver, the run script must be modified manually. The information about the available connectors is available at reports/connectors. The framebuffer configuration can be edited in config/fb_drv.config.
• The audio driver and mixer are integrated. VirtualBox connects to the mixer so that music can be played from within the Rich OS. For changing the master volume, edit config/mixer.config.
• A backdrop has been added, which can be edited via config/backdrop.config.

[nfeske]

Please note the fixup https://github.com/nfeske/genode/commit/8c884445268bed7674fecee26c7a7da44b9bc1b2 commit on my branch.

[nfeske]

I rebased my turmvilla branch to the current master. The most noteworthy change is the principal support for manipulating windows with the keyboard. The maximized state can be toggled via win+enter, the focus can be switched between the most recently focused windows via win+tab and win+shift+tab.

[ohpauleez]

Just advertising a fork of the turmvilla branch with a Vagrant setup already in place, here.

This automates the entire process: Creating a Linux build VM with a shared directory, setting up all dependencies, performing the build of a the Turmvilla Scenario (turmvilla with NOVA and all repositories), producing the final ISO /var setup.

I'm more than happy to contribute any of the work back.

[ssumpf]

@ohpauleez: :+1:

[m-stein]

@ohpauleez That sounds very cool! I'll have a try when I'm back in the office next week.

[nfeske]

I rebased my turmvilla branch to Genode 16.02. The most visible improvement is the support for assigning USB devices to the rich OS.

Once a new USB device is plugged in, the user can look at /reports/usb_devices to obtain the device's vendor and device IDs. Those information can then be used to add an entry to the /config/usb_report_filter.config. When the device is plugged-in the next time, the USB driver will automatically be reconfigured with the policy needed to assign the device to the Linux VM. The USB-driver configuration can be inspected at /config/usb_drv.config.

[ohpauleez]

I have a few changes on my fork I'll commit soon: I automated the last steps with sed, so that a vagrant up automatically produces the iso in the shared directory when initially launched. I'm also happy to port the Vagrantfile over to a Dockerfile, providing more options for users/developers.

[nfeske]

Turmvilla won't receive any updates.