OpenBLAS WARNING when starting R in rocker containers managed by rancher desktop

erblast commented 2 years ago

Actual Behavior

This might be a very specific issue. But I could not find a solution for it anywhere, so I am posting it here.

When starting R in in any rocker container (which is the most popular docker container distribution for R and RStudio) using rancher desktop I get this warning on start up.

OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k

It does not effect R but it breaks unit tests for R packages because it throws a warning causing the overall tests to fail. This does not happen in docker desktop.

Steps to Reproduce

docker run -d rocker/rstudio:latest
docker container ls
docker container exec -it {8e565d181f77} /bin/bash
R

Result

OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k

Expected Behavior

Additional Information

No response

Rancher Desktop Version

1.2.1

Rancher Desktop K8s Version

v1.22.6 (but kubernetes is not enabled)

Which container runtime are you using?

moby (docker cli)

What operating system are you using?

macOS

Operating System / Build Version

Catalina 10.15.7

What CPU architecture are you using?

x64

Linux only: what package format did you use to install Rancher Desktop?

No response

Windows User Only

No response

gunamata commented 2 years ago

@jandubois , Is it possible to set L2 Cache size via override.yaml?

mook-as commented 2 years ago

The relevant code appears to be: https://github.com/xianyi/OpenBLAS/blob/34ecd967a54f71cb10fb07bf321b2a7597db9d26/kernel/setparam-ref.c#L1200

It's probably another missing cpuid instruction?

jandubois commented 2 years ago

It's probably another missing cpuid instruction?

No, QEMU does not pass through the host cache info by default. I think you can enable it with something like

-cpu host,host-cache-info=on

But I would like to understand why it is disabled by default (the info may be wrong for certain CPUs with shared caches because the CPUs used by the VM may not correspond to the host CPUs). It also looks like the property is not exposed on all CPUs, so indiscriminately adding it to the commandline might cause errors on certain CPU models.

Is it possible to set L2 Cache size via override.yaml

No, it is not. The only way to affect this would be setting an environment variable in the context of the Rancher Desktop app:

QEMU_SYSTEM_X86_64="qemu-system-x86_64 -cpu host,host-cache-info=on"

Note that RD is started via launchd, so does not inherit the user's shell environment settings. You could probably add it to /Applications/Rancher\ Desktop.app/Contents/Info.plist, but I haven't tried it yet.

I also don't know if the directory containing qemu-system-x86_64 is on the PATH at the time limactl start is executed. If not, then the environment variable would need to include the full path:

/Applications/Rancher\ Desktop.app/Contents/Resources/resources/darwin/lima/bin/qemu-system-x86_64

And I don't know if quoting will work here to deal with the embedded space in the directory name. I'll see if I can find some time later to actually try it...

jandubois commented 2 years ago

I've written a script to edit the Info.plist file and add the variable:

#!/bin/sh

set -eux

PLISTBUDDY="/usr/libexec/PlistBuddy"
INFO_PLIST="/Applications/Rancher Desktop.app/Contents/Info.plist"

NAME="QEMU_SYSTEM_X86_64"
VALUE="qemu-system-x86_64 -cpu host,host-cache-info=on"

"${PLISTBUDDY}" -c "Add :LSEnvironment dict" "${INFO_PLIST}"
"${PLISTBUDDY}" -c "Add :LSEnvironment:${NAME} string" "${INFO_PLIST}"
"${PLISTBUDDY}" -c "Set :LSEnvironment:${NAME} \"$VALUE\"" "${INFO_PLIST}"
"${PLISTBUDDY}" -c "Print :LSEnvironment" "${INFO_PLIST}"

And it seems to work:

$ ./info-env.sh
+ PLISTBUDDY=/usr/libexec/PlistBuddy
+ INFO_PLIST='/Applications/Rancher Desktop.app/Contents/Info.plist'
+ NAME=QEMU_SYSTEM_X86_64
+ VALUE='qemu-system-x86_64 -cpu host,host-cache-info=on'
+ /usr/libexec/PlistBuddy -c 'Add :LSEnvironment:QEMU_SYSTEM_X86_64 string' '/Applications/Rancher Desktop.app/Contents/Info.plist'
+ /usr/libexec/PlistBuddy -c 'Set :LSEnvironment:QEMU_SYSTEM_X86_64 "qemu-system-x86_64 -cpu host,host-cache-info=on"' '/Applications/Rancher Desktop.app/Contents/Info.plist'
+ /usr/libexec/PlistBuddy -c 'Print :LSEnvironment' '/Applications/Rancher Desktop.app/Contents/Info.plist'
Dict {
    QEMU_SYSTEM_X86_64 = qemu-system-x86_64 -cpu host,host-cache-info=on
}

Unfortunately the setting was not applied to the Rancher Desktop application.

I then told launchd to add the setting to every process it starts:

$ /bin/launchctl setenv QEMU_SYSTEM_X86_64 "qemu-system-x86_64 -cpu host,host-cache-info=on"

And that took effect:

$ ps -ef | grep host-cache
  501 63664 63653   0  4:02pm ??         2:55.83 /Applications/Rancher Desktop.app/Contents/Resources/resources/darwin/lima/bin/qemu-system-x86_64 -cpu host,host-cache-info=on -m 6144 -machine q35,accel=hvf -smp 2,sockets=1,cores=2,threads=1 -boot order=d,splash-time=0,menu=on -drive file=/Users/jan/Library/Application Support/rancher-desktop/lima/0/basedisk,media=cdrom,readonly=on -drive file=/Users/jan/Library/Application Support/rancher-desktop/lima/0/diffdisk,if=virtio -cdrom /Users/jan/Library/Application Support/rancher-desktop/lima/0/cidata.iso -netdev user,id=net0,net=192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:52521-:22 -device virtio-net-pci,netdev=net0,mac=52:55:55:de:ac:46 -netdev vde,id=net1,sock=/private/var/run/rancher-desktop-shared.ctl -device virtio-net-pci,netdev=net1,mac=52:55:55:06:42:0a -netdev vde,id=net2,sock=/private/var/run/rancher-desktop-bridged_en0.ctl -device virtio-net-pci,netdev=net2,mac=52:55:55:e4:bb:a3 -device virtio-rng-pci -display none -device virtio-vga -device virtio-keyboard-pci -device virtio-mouse-pci -parallel none -chardev socket,id=char-serial,path=/Users/jan/Library/Application Support/rancher-desktop/lima/0/serial.sock,server=on,wait=off,logfile=/Users/jan/Library/Application Support/rancher-desktop/lima/0/serial.log -serial chardev:char-serial -chardev socket,id=char-qmp,path=/Users/jan/Library/Application Support/rancher-desktop/lima/0/qmp.sock,server=on,wait=off -qmp chardev:char-qmp -name lima-0 -pidfile /Users/jan/Library/Application Support/rancher-desktop/lima/0/qemu.pid

Unfortunately that didn't help with the cache detection:

$ docker container exec -it keen_hypatia /bin/bash
root@39505242679d:/# R
OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k

R version 4.2.0 (2022-04-22) -- "Vigorous Calisthenics"

I'm out of time to spend on this now, but hopefully this provides enough information for somebody else to perform further research and experiments.

I'll leave this issue open for now, but I don't think this is really a Rancher Desktop problem. Everything is working correctly. It is just that OpenBLAS displays a warning on stderr on certain virtual machine configurations.

rancher-sandbox / rancher-desktop