omap_aes and dm-crypt problems

[birefringence]

I'm trying to use dm-crypt with AES HW acceleration on the Beagle Bone Black using the Ubuntu Precise image running from microSD card with an updated ubuntu-armhf 3.8.13-bone28 kernel.

Unfortunately, I get a whole lot of these (e.g. when running "cryptsetup luksFormat" or any other LUKS operation I have tried):

[ 4978.811789] BUG: scheduling while atomic: swapper/0/0/0x40000100
[ 4978.818085] Modules linked in: sha256_generic dm_crypt
[ 4978.818116] [<c0013250>] (unwind_backtrace+0x0/0xe0) from [<c060f804>] (__schedule_bug+0x48/0x5c)
[ 4978.818134] [<c060f804>] (__schedule_bug+0x48/0x5c) from [<c0615704>] (__schedule+0x68/0x6dc)
[ 4978.818151] [<c0615704>] (__schedule+0x68/0x6dc) from [<c00653fc>] (__cond_resched+0x24/0x34)
[ 4978.818166] [<c00653fc>] (__cond_resched+0x24/0x34) from [<c0615e04>] (_cond_resched+0x34/0x44)
[ 4978.818184] [<c0615e04>] (_cond_resched+0x34/0x44) from [<c038d548>] (__pm_runtime_idle+0x2c/0x94)
[ 4978.818203] [<c038d548>] (__pm_runtime_idle+0x2c/0x94) from [<c04be718>] (omap_aes_finish_req+0x1c/0x3c)
[ 4978.818220] [<c04be718>] (omap_aes_finish_req+0x1c/0x3c) from [<c04beb68>] (omap_aes_done_task+0x19c/0x1c4)
[ 4978.818239] [<c04beb68>] (omap_aes_done_task+0x19c/0x1c4) from [<c0042f38>] (tasklet_action+0x7c/0xd8)
[ 4978.818255] [<c0042f38>] (tasklet_action+0x7c/0xd8) from [<c00426c4>] (__do_softirq+0x110/0x23c)
[ 4978.818271] [<c00426c4>] (__do_softirq+0x110/0x23c) from [<c0042aa0>] (irq_exit+0x44/0x84)
[ 4978.818287] [<c0042aa0>] (irq_exit+0x44/0x84) from [<c000de5c>] (handle_IRQ+0x68/0x8c)
[ 4978.818302] [<c000de5c>] (handle_IRQ+0x68/0x8c) from [<c0008754>] (omap3_intc_handle_irq+0x60/0x74)
[ 4978.818317] [<c0008754>] (omap3_intc_handle_irq+0x60/0x74) from [<c0616ec0>] (__irq_svc+0x40/0x50)
[ 4978.818326] Exception stack(0xc0939f28 to 0xc0939f70)
[ 4978.818338] 9f20:                   00000037 00000000 c0a215cc 00000294 df31b64c df2f519c
[ 4978.818351] 9f40: df2f518c 00000000 80004059 413fc082 00000000 00000000 00000000 c0939f70
[ 4978.818362] 9f60: c04b9d50 c030be00 60000013 ffffffff
[ 4978.818379] [<c0616ec0>] (__irq_svc+0x40/0x50) from [<c030be00>] (__gpio_set_value+0x10/0xfc)
[ 4978.818396] [<c030be00>] (__gpio_set_value+0x10/0xfc) from [<c04b9d50>] (led_set_brightness+0x54/0x58)
[ 4978.818412] [<c04b9d50>] (led_set_brightness+0x54/0x58) from [<c04ba280>] (led_trigger_event+0x30/0x48)
[ 4978.818427] [<c04ba280>] (led_trigger_event+0x30/0x48) from [<c000e168>] (cpu_idle+0x30/0xf0)
[ 4978.818445] [<c000e168>] (cpu_idle+0x30/0xf0) from [<c08ee7a4>] (start_kernel+0x2bc/0x314)

Other people seem to have a similar experience with the SHA acceleration: http://osdir.com/ml/beagleboard/2013-07/msg01375.html

[RobertCNelson]

How does it work with 3.12? https://github.com/RobertCNelson/linux-dev/tree/am33x-v3.12

Regards,

[birefringence]

Well ... I haven't succeeded to get the precompiled 3.12 Kernel to boot on my Precise image. So I took this rootfs: https://rcn-ee.net/deb/rootfs/raring/ubuntu-13.04-console-armhf-2013-09-26.tar.xz and installed the 3.12 Kernel with http://rcn-ee.net/deb/raring-armhf/v3.12.0-rc5-bone6/install-me.sh but it does not include the dm-crypt module ... So I guess I need to compile the Kernel myself. This will take a couple of days until I have the time to get it done.

[masand]

Hi, ran into the same problem. There were loads of exceptions when trying anything with LUKS. Took me quite some time to find this discussion, which is the only hint I got so far.

I can confirm that the problem exists in the images for quantal, raring and saucy (didn´t check again, but I guess the kernel is the same for all).

I´m about to compile kernel 3.12 right now. Will report the result in some hours.

[masand]

Hi again, with kernel 3.12, this is working fine! I could encrypt a USB stick with LUKS, save some files to it, put the stick into another computer, decrypt them there and verify there the files are still exactly the same.

Though this was quite some patchwork to get the kernel onto the beagle bone. I used the setup_sdcard.sh script and added the files needed there (xxx-dtbs.tar.gz, initrd.xxx, uInitrd-xxx, vmlinuz-xxx) from different places after compiling the kernel. The kernel was compiled (crosscompiled) on my PC, and also on the beaglebone itself in parallel. I can´t reconstruct any more what I did exactly, but I found an initrd.xxx only on the beaglebone and hence copied it from there... (uggh...) Sorry, the times where I had more knowledge about compiling kernels are some years ago...

So though this exceeds the topic of this discussion: Is there a proper description how to build and install a new kernel for Ubuntu on the beagle bone? Where the kernel is at least installed (not necessarily build) from "within" the beagle bone? Or a script for that? All I could find where descriptions for Angstrom, or Debian specific, or in conjunction with the getting the kernel to the sdcard from the "outside", not via the beagle bone. This had the drawback to start at zero with your system configuration.

Thanks in advance.

[RobertCNelson]

For Ubuntu/Debian microSD card's created with "setup_sdcard.sh" just use the "./tools/install_kernel.sh" script in my kernel build scripts.. https://github.com/RobertCNelson/linux-dev/blob/am33x-v3.12/README#L34

Whereas "initrd.xyz" has to be generated "ON" target.. So you can either do it yourself, or run teh "/boot/uboot/tools/update_boot_files.sh" script "setup_sdcard.sh" also populates.

[birefringence]

Yep, with 3.12 the errors are gone.

[birefringence]

Unfortunately I have to reopen this: While there is no error anymore, I also have some serious doubts that the hardware acceleration is actually used.

I tried to check performance using this guide http://processors.wiki.ti.com/index.php/AM335x_Crypto_Performance#DM_Crypt_Performance_2

and I got cached reads of about 190MB/s and buffered reads of about 12 MB/s.

In addition to be very close to the non-accelerated speed from the link above, this number did not change independent of whether omap_aes and omap_sham were compiled in, were loaded as a module or were not loaded at all. However, the number dropped even further to 9 MB/s when disabling the ARM-asm AES algorithm from the Kernel config, so I assume the software AES is used in all cases.

By the way, the cached reads from a ram disk are faster (250MB/s compared to 210 MB/s) when using the 3.8.13-bone28 Kernel (of course, without dm-crypt).

I also noticed, that USB performance is quite bad (I should probably open a separate issue for this). Running hdparm -tT on a USB 2.0 thumb drive gives a buffered read speed of 9MB/s, while on other systems (e.g. a Raspberry Pi) the same command on the same thumb drive gives a speed of 24MB/s.

I was hoping the BBB to outperform my Raspberry Pi when reading from encrypted media over USB, due to the vastly better specs. But this currently does not seem to be the case ...

[RobertCNelson]

This really the wrong list, please contact the crypto maintainers..

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/drivers/crypto/omap-aes.c

Looks like Joel was the last to work on this.. Regards,

[masand]

Last week, I tested the performance of writing to a LUKS (AES) encrypted USB hard disk with EXT4 at both the raspberry pi and the beaglebone. The results were about 1MB/s for the raspberry pi and 3MB/s for the beaglebone. Didn´t test reading, though.

Please keep me posted where this discussion continues.

[birefringence]

Ok, thanks for the pointer! Shall I close this issue then here?

Do you also have a suggestion whom I could contact because of the USB performance issue?

[birefringence]

I now sent an e-mail to Joel. We should move the discussion to the BeagleBoard Google group. I will post a topic (and a link to it here) once I have any new information.

@masand: My overclocked Raspberry (950Mhz) and the BBB both currently achieve about 6MB/s reading from AES encrypted file systems over USB.

[masand]

Can acknowledge the 6MB/s for AES encrypted file system via USB. What values would you expect? Any progress on the topic?

[birefringence]

Well, the AM335x Linux PSP with Kernel 3.2 from TI claims much higher speeds, so I'm currently setting it up. If I take these http://processors.wiki.ti.com/index.php/AM335x_Crypto_Performance#DM_Crypt_Performance_2 and these http://processors.wiki.ti.com/index.php/AM335x-PSP_04.06.00.11_Features_and_Performance_Guide#USB_Driver numbers I would expect that at least twice as much speed as I'm seeing currently is possible. The simplest way to get this running seems to be via the Sitara SDK. Unfortunately, I have to wait for the required FTDI serial cable to be delivered.

In case it does work better with the TI Kernel, I will try to find out what is the difference.

[birefringence]

FTDI cable has arrived today. The sub-par USB performance is gone with the TI kernel, haven't gotten to run any crypto yet (looks like I need to compile some stuff).

[birefringence]

Please follow me here for further discussions: http://beagleboard.org/Community/Forums?place=topic%2Fbeagleboard%2FtZ5esfCaH3Q%2Fdiscussion

I will close this issue until I know what's going on and have some constructive ideas to contribute.