mviereck / x11docker

Run GUI applications and desktops in docker and podman containers. Focus on security.
MIT License
5.62k stars 378 forks source link

GPU support on arm / RaspBerry Pi #105

Closed mviereck closed 5 years ago

mviereck commented 5 years ago

Coming from #100:

Basically x11docker runs on a RaspBerry Pi. For advanced support of RaspBerry hardware the image needs package libraspberrypi0. Some issues remain, e.g. how to allow GPU access without --privileged, and choppy videos with kodi in container.

Citing a summary from @1138-4EB:

resin/armv7hf-debian

It is based on Jessie, so kodi is not available.

balenalib/armv7hf-debian

root@ad30d9a3384c:~# kodi
/usr/lib/arm-linux-gnueabihf/kodi/kodi-xrandr: Failed to get size of gamma for output default
libEGL warning: DRI2: failed to authenticate
libEGL warning: DRI2: failed to open swrast (search paths /usr/lib/arm-linux-gnueabihf/dri:${ORIGIN}/dri:/usr/lib/dri)
terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string::_M_construct null not valid
Aborted (core dumped)
Crash report available at /fakehome/root/kodi_crashlog-20190120_232325.log

root@ad30d9a3384c:~# apt install libraspberrypi0 -y
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package libraspberrypi0

kodi is available, but it does not work. libraspberrypi0 is not available.

raspbian/stretch

/usr/bin/kodi: 1: /usr/bin/kodi: fbset: not found
/usr/bin/kodi: 46: /usr/bin/kodi: tvservice: not found
/usr/bin/kodi: 60: /usr/bin/kodi: tvservice: not found

Both kodi and libraspberrypi0 are available.

It complains about fbset and tvservice, but it works.

schachr/raspbian-stretch

/usr/lib/arm-linux-gnueabihf/kodi/kodi-xrandr: Failed to get size of gamma for output default
libEGL warning: DRI2: failed to authenticate

kodi is available, and it does work. libraspberrypi0 is not available.

armv7/debian

root@0358016a4b96:~# kodi
/usr/lib/arm-linux-gnueabihf/kodi/kodi-xrandr: Failed to get size of gamma for output default
libEGL warning: DRI2: failed to authenticate
/usr/lib/arm-linux-gnueabihf/kodi/kodi-xrandr: Failed to get size of gamma for output default

kodi is available, and it does work. libraspberrypi0 is not available.


Little update. I could install libraspberrypi0 on armv7/debian:

Add these to /etc/apt/sources.list:

deb http://archive.raspbian.org/raspbian stretch main contrib non-free rpi firmware
deb http://archive.raspberrypi.org/debian stretch main

Then:

apt install -y wget gnupg
wget https://archive.raspbian.org/raspbian.public.key -O - | apt-key add -
wget http://archive.raspberrypi.org/debian/raspberrypi.gpg.key -O - | apt-key add -
apt update -y
apt install -y libraspberrypi0

However, kodi is still using the CPU. I tried adding the packages mentioned in hypriot/x11-on-HypriotOS#2 too. Still the same. I think I am missing some command to reload something after the packages are installed.

lpt2007 commented 5 years ago

What about this errors:

debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/arm-linux-gnueabihf/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/arm-linux-gnueabihf/perl5/5.24 /usr/share/perl5 /usr/lib/arm-linux-gnueabihf/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/arm-linux-gnueabihf/perl-base .) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7, <> line 21.)
debconf: falling back to frontend: Teletype
dpkg-preconfigure: unable to re-open stdin:

Error shows with red text when I run docker build somewhere in middle in proces. Could this error cause choppy videos?

mviereck commented 5 years ago

Error shows with red text when I run docker build somewhere in middle in proces.

Those messages are harmless. You can avoid them if you insert a line ENV DEBIAN_FRONTEND=noninteractive after the FROM line.

Possible attempts for investigation:

lpt2007 commented 5 years ago

I try tu use 64bit sistem to see if any difference.

I flashed this image: pi64-lite.zip https://github.com/bamarni/pi64/releases

After sistem stareted I updatete and upgrade sistem:

apt-get update
apt-get upgrade

then I installed docker:

apt-get install curl
curl -fsSL get.docker.com -o get-docker.sh && sh get-docker.sh
then I installed x11docker:
apt-get install wget
wget https://raw.githubusercontent.com/mviereck/x11docker/master/x11docker -O /tmp/x11docker
sudo bash /tmp/x11docker --update
rm /tmp/x11docker

then I creted dockerfile

FROM raspbian/stretch
RUN apt-get update
RUN apt-get upgrade -y
RUN apt-get install -y kodi
CMD kodi

build and run kodi:

sudo docker build -t kodi /home/pi/docker/kodi
x11docker kodi

or x11docker --gpu -- --privileged -- kodi

error kodi won't start:

/usr/bin/kodi: 1: /usr/bin/kodi: fbset: not found
/usr/bin/kodi: 46: /usr/bin/kodi: tvservice: not found
/usr/bin/kodi: 77: /usr/bin/kodi: arithmetic expression: expecting ')': "(0x >> 23) & 1"

fbset and tvservice are know errors, but arithmetic expression is new.

What arithmetic expression error means?

mviereck commented 5 years ago

What arithmetic expression error means?

It looks like a bug in kodi itself. However, it is surprising that it didn't occur before. Your new host system would not explain that, because the bug occcurs within the container. I don't know a reason for this bug. You can try to run kodi with command kodi-standalone instead of kodi. Maybe it helps.

lpt2007 commented 5 years ago

I try with kodi-standalone but still not working.

Same error.

eine commented 5 years ago

It might be worth trying with some arm64v8 image.

lpt2007 commented 5 years ago

I try with arm64v8/debian and I get error:

xcb_connection_has_error() returned true
ERROR: Unable to create GUI. Exiting
Aborted (core dumped)
mviereck commented 5 years ago

xcb_connection_has_error()

I am not sure, but it might be related to #76. Try to run with --security-opt seccomp=unconfined, e.g.

 x11docker --gpu -- --privileged --security-opt seccomp=unconfined -- kodi
lpt2007 commented 5 years ago

Ater I run: x11docker --gpu -- --privileged --security-opt seccomp=unconfined -- kodi

Error:

xcb_connection_has_error() returned true
ERROR: Unable to create GUI. Exiting
Aborted (core dumped)
Crash report available at /fakehome/pi/kodi_crashlog-20190127_122249.log
mviereck commented 5 years ago

It makes sense to divide this ticket. This one should contain only issues concerning GPU on Raspberry Pi / arm, maybe also general issues that are special to Raspberry/arm hardware.

The issues you describe look rather kodi specific. Please open a new ticket for issues to get kodi to run at all on Raspberry Pi in a container. E.g. call it "Issues with kodi in container on RaspBerry Pi".

lpt2007 commented 5 years ago

But If I run container on 32bit host sistem kodi start ok, but I get choppy video?

lpt2007 commented 5 years ago

I try on diet-pi image.

Error:

/usr/bin/kodi: 1: /usr/bin/kodi: fbset: not found
/usr/bin/kodi: 46: /usr/bin/kodi: tvservice: not found
* failed to add service - already in use?
Aborted (core dumped)
Crash report available at /fakehome/root/kodi_crashlog-20190127_135422.log
/usr/bin/kodi: 60: /usr/bin/kodi: tvservice: not found
x11docker error: Did not find a possibility to provide a display.
Recommendation:
To run within an already running X server, install one or all of:
Xephyr xpra nxagent
To run with GPU acceleration, install:
weston and Xwayland, optionally also: xpra and xdottool
To run from TTY or within Wayland, install:
weston and Xwayland
lpt2007 commented 5 years ago

I try under alpine linux.

Is working like on raspbian host(choppy vdeo).

Only difference is that keyboard is working.

eine commented 5 years ago

I try under alpine linux.

Is working like on raspbian host(choppy vdeo).

Only difference is that keyboard is working.

Which packages did you install in the alpine linux image? Did it require anything similar to libraspberrypi0?

lpt2007 commented 5 years ago

Which packages did you install in the alpine linux image? Did it require anything similar to libraspberrypi0? No.

I just enabled openGL on alpine linux host, informaton is on this page: https://wiki.alpinelinux.org/wiki/Raspberry_Pi

mviereck commented 5 years ago

I try under alpine linux. Is working like on raspbian host(choppy vdeo). Only difference is that keyboard is working. I just enabled openGL on alpine linux host,

That sounds good so far.

To see if bugs are related to kodi only or to Raspberry hardware instead, please always install vlc media player in image, too. If somethings works with vlc, but not with kodi, it is a kodi issue. If something fails with both of them, it is a general issue.

Is your host alpine system 32bit or 64bit? Is the image system 32bit or 64bit? What is your current Dockerfile?

For now please always run with --privileged to exclude possible issues with hardware access permissions.

lpt2007 commented 5 years ago

Show me how to install vlc and I will try.

Is the image system 32bit or 64bit? What is your current Dockerfile? System is 32bit.

Dockerfile:

FROM raspbian/stretch
RUN apt-get update
RUN apt-get upgrade -y
RUN apt-get install -y libraspberrypi0
RUN apt-get install -y kodi
CMD kodi
mviereck commented 5 years ago

Show me how to install vlc and I will try.

Add a line in Dockerfile and rebuild:

RUN apt-get install -y vlc

Run image with:

 x11docker --gpu -- --privileged -- kodi vlc
eine commented 5 years ago

It looks like a bug in kodi itself. However, it is surprising that it didn't occur before. Your new host system would not explain that, because the bug occcurs within the container.

The error might be related to the fact that he was running a 32-bit container on a 64-bit system.

I just enabled openGL on alpine linux host, informaton is on this page: wiki.alpinelinux.org/wiki/Raspberry_Pi

In order to reproduce and hopefully fix the issues, we need you to be more specific when posting the results of the tests you run. I just enabled openGL is not very helpful. In the link you provide, we see in section Enable OpenGL (RPi 3) that some drivers are installed (mesa-dri-vc4). However, we must assume that you have a running X server too. Hence, you probably also installed some of the packages in section X11 Setup: setup-xorg-base ​apk add xf86-video-fbdev xf86-video-vesa xf86-input-mouse xf86-input-keyboard dbus ​set​xkbmap kbd rc-update ​​add dbus (xfce4 too?). Can you confirm this? Did you install any additional package?

Certainly, this would explain why the keyboard is working. Note that packages xf86-input-keyboard, xkbmap and kbd are installed. Some of these might be missing in the previous setup you tried.

Furthermore, after installing the packages lbu_commit -d is executed. We commented before that we might be missing the execution of some command to apply changes after installing packages/drivers. Might this be it?

The bottom line is that we can try installing the same packages in a alpine container (see docker manifest inspect alpine or https://hub.docker.com/_/alpine). There are two variants for ARM: ARMv6 (32-bit) and ARMv8 (64-bit). I would recommend the later, but since you seem to be using the 32-bit version, you can try with it.

Show me how to install vlc and I will try.

apt install -y vlc

System is 32bit.

As commented above, I suggest trying alpine-rpi-3.8.2-aarch64.tar.gz instead of alpine-rpi-3.8.2-armhf.tar.gz.

So, it seems that you are running docker on a 32bit Alpine Linux host, and the container you are executing is the same that worked with raspbian as a host.

I think that the issue here is not related to the host system, as long as the app works as expected natively. Since the app is choppy inside the container and not outside, we should focus on the content of the docker image.

I assume that you have tried executing kodi natively on Alpine Linux. If that worked, and the container based on raspbian is still choppy, we might be lucky. We have not guessed the differences between the Raspbian in the SD and the raspbian container, but we might achieve it with Alpine. As suggested above, I'd try installing in a alpine docker image the same packages you installed on the host.

mviereck commented 5 years ago

Unfortunately alpine does not provide a kodi package for arm architecture but x86/x86_64 only. We can either go on with current setup (host: alpine, image: raspian), or @lpt2007 returns to raspian/stretch for both host and image.

Edit: In a previous test with raspian on host kodi had no issues on host. We might focus on missing packages in the image as you said.

lpt2007 commented 5 years ago

Hence, you probably also installed some of the packages in section X11 Setup: setup-xorg-base ​apk add xf86-video-fbdev xf86-video-vesa xf86-input-mouse xf86-input-keyboard dbus ​set​xkbmap kbd rc-update ​​add dbus (xfce4 too?). Can you confirm this? Did you install any additional package?

Yes I installed those packages.

lpt2007 commented 5 years ago

Hi, I returned to host:raspbian and image:raspbina and I try to install from scratch from raspbian-lite image from here: https://www.raspberrypi.org/downloads/raspbian/

I enabled ssh with:

sudo apt-get update
sudo systemctl enable ssh
sudo reboot

I update host system:

sudo apt-get upgrade -y

run raspi-config: sudo raspi-config

go to -> "Interfacing Options" -> "Camera" -> Enable, go to -> "Advanced Options" -> "Memory Split" -> 256 go to -> "Finish" -> "Reboot"

I installed docker:

curl -fsSL get.docker.com -o get-docker.sh && sh get-docker.sh
sudo usermod -aG docker pi

I Installed x11docker

sudo apt-get install -y xdg-utils
wget https://raw.githubusercontent.com/mviereck/x11docker/master/x11docker -O /tmp/x11docker
sudo bash /tmp/x11docker --update
rm /tmp/x11docker

I create folders:

mkdir /home/pi/docker
mkdir /home/pi/docker/kodi

create and edit Dockerfile: nano /home/pi/docker/kodi/Dockerfile add this:

FROM raspbian/stretch
RUN apt-get update && apt-get dist-upgrade -y
RUN apt-get install -y fbset libraspberrypi0 kodi vlc
CMD kodi

build docker container: sudo docker build -t kodi /home/pi/docker/kodi

from ssh I run this:

export DISPLAY=:0
x11docker --homedir /its/docker/kodi --gpu -- --privileged -- kodi

When kodi starts: go to -> "system" -> "player settings - Expert" -> check "Allow hardware acceleration - OMXPlayer"

Now moves are playing ok. :)

todo: keyboard not working (only remote over cec)

lpt2007 commented 5 years ago

I try to change potions under: sudo raspi-config

under Advanced options -- GL Driver is G1 GL (Full KMS) OpenGL desktop driver with full KMS kodi won't start G2 GL (Fake KMS) OpenGL desktop driver with fake KMS kodi won't start G3 Legacy Original non-GL desktop driver kodi start with choppy video problem

lpt2007 commented 5 years ago

Which X11 packages must be installed?

Which packages must be installed for keyboard?

lpt2007 commented 5 years ago

I found some info about kodi on raspbian: https://www.raspberrypi.org/forums/viewtopic.php?t=192499

lpt2007 commented 5 years ago

I finally made kodi that works under docker with working video :). Look four posts up.

Now I have further questions.

It is possible to start multi docker kodi containers (container with kodi 17.6 and container kodi 18) and then just switch between them like channels on tv?

mviereck commented 5 years ago

I finally made kodi that works under docker with working video :).

Great! What have been the missing key to fix the choppy videos?

It is possible to start multi docker kodi containers (container with kodi 17.6 and container kodi 18) and then just switch between them like channels on tv?

You can run a slim desktop on the host, e.g. lxde (or openbox only). There you can run multiple instances of x11docker+kodi and switch between them with Alt+Tab.

lpt2007 commented 5 years ago

Great! What have been the missing key to fix the choppy videos?

I think problem is memory split setthings and check "Allow hardware acceleration - OMXPlayer"

You can run a slim desktop on the host, e.g. lxde (or openbox only). There you can run multiple instances of x11docker+kodi and switch between them with Alt+Tab

Can this been done through script?

Now I already trying to make it work on hassos 64bit image. I installed x11docker and build docker container without problem. Now I have problem with read-only cache directory error. /root/.cache read-only file sistem

ConstantinFF commented 5 years ago

Which X11 packages must be installed?

Which packages must be installed for keyboard?

Hey, currently I am struggling with the same keyboard/mouse issue. Did you make it to run?

mviereck commented 5 years ago

I hope @lpt2007 will give you an answer. I admit I am confused from the multiple different setups he has tried, and cannot tell which one works indeed.

ConstantinFF commented 5 years ago

Hi @lpt2007 Did you manage to fix the keyboard issue?

lpt2007 commented 5 years ago

Sorry for so late answer:(

Here is working version: LINK