search

void-linux / void-packages

The Void source packages collection
https://voidlinux.org
Other
2.52k stars 2.12k forks source link

Waydroid shipped broken #41363

Open notramo opened 1 year ago

notramo commented 1 year ago

Is this a new report?

Yes

System Info

Void x64 musl

Package(s) Affected

waydroid

Does a report exist for this bug with the project's home (upstream) and/or another distro?

No response

Expected behaviour

Waydroid starts, or the guide is complete.

Actual behaviour

Waydroid doesn't start. The attached guide (README.voidlinux) is incomplete. waydroid log includes the following error message:

dnsmasq: failed to open pidfile /run/waydroid-lxc/dnsmasq.pid: Permission denied

Steps to reproduce

  1. Install waydroid
  2. Follow the steps in /usr/share/doc/waydroid/README.voidlinux
  3. Add the psi=1 parameter to GRUB config and reboot
  4. Start the system service.
  5. waydroid session start
  6. waydroid log to view the error
paper42 commented 1 year ago

@JamiKettunen

JamiKettunen commented 1 year ago

@notramo I couldn't reproduce this in a fresh Void x86_64-musl VM, weston-x11 launched as the compositor on Xfce4 image copied from ISO to disk and updated, and https://docs.waydro.id/faq/get-waydroid-to-work-through-a-vm done to get graphical output since glxinfo | grep 'OpenGL renderer' showed llvmpipe.

What does waydroid status show? There should be A LOT more details in waydroid log than that either way, run it before waydroid session start. You could possibly even down waydroid-container service and start it after starting to tail the log too to get even more details.

Curious if this could be a clash with another service trying to use some same address etc, perhaps ls /var/service would be useful (assuming you don't start system services elsewhere).

Btw waydroid session start also won't show a GUI, see waydroid show-full-ui instead for example. .desktop entries for Waydroid and the installed apps should also be created assuming you have an app launcher iterating those.

notramo commented 1 year ago

Full log (waydroid log started before launching container):

(013323) [22:18:55] % modprobe -q ashmem_linux
(013323) [22:18:55] % chmod 666 -R /dev/anbox-binder
(013323) [22:18:55] % chmod 666 -R /dev/anbox-vndbinder
(013323) [22:18:55] % chmod 666 -R /dev/anbox-hwbinder
(013323) [22:18:55] Container manager is waiting for session to load
(013716) [22:19:12] Save session config: /var/lib/waydroid/session.cfg
(013716) [22:19:12] UserMonitor service is not even started
(013716) [22:19:12] Clipboard service is not even started
(013323) [22:19:12] % /usr/lib/waydroid/data/scripts/waydroid-net.sh start
vnic is waydroid0

dnsmasq: failed to open pidfile /run/waydroid-lxc/dnsmasq.pid: Permission denied
Failed to setup waydroid-net.
Failed to setup waydroid-net.
(013323) [22:19:12] % umount /var/lib/waydroid/rootfs/vendor/waydroid.prop
(013323) [22:19:12] % umount /var/lib/waydroid/rootfs/vendor
(013323) [22:19:12] % umount /var/lib/waydroid/rootfs
(013323) [22:19:12] % mount /var/lib/waydroid/images/system.img /var/lib/waydroid/rootfs
(013323) [22:19:13] % mount -o remount,ro /var/lib/waydroid/images/system.img /var/lib/waydroid/rootfs
(013323) [22:19:13] % mount /var/lib/waydroid/images/vendor.img /var/lib/waydroid/rootfs/vendor
(013323) [22:19:13] % mount -o remount,ro /var/lib/waydroid/images/vendor.img /var/lib/waydroid/rootfs/vendor
(013323) [22:19:13] % mount -o bind /var/lib/waydroid/waydroid.prop /var/lib/waydroid/rootfs/vendor/waydroid.prop
(013323) [22:19:13] Save config: /var/lib/waydroid/waydroid.cfg
(013323) [22:19:13] % chmod 777 -R /dev/dri
(013323) [22:19:13] % chmod 777 -R /dev/fb0
(013323) [22:19:13] % chmod 777 -R /dev/video1
(013323) [22:19:13] % chmod 777 -R /dev/video0
(013323) [22:19:13] % lxc-start -P /var/lib/waydroid/lxc -F -n waydroid -- /init
(013323) [22:19:13] New background process: pid=13787, output=background
(013323) [22:19:13] waiting 10 seconds for container to start...
lxc-start: waydroid: ../src/lxc/network.c: netdev_configure_server_veth: 711 No such file or directory - Failed to attach "vethrKzhjZ" to bridge "waydroid0", bridge interface doesn't exist
lxc-start: waydroid: ../src/lxc/network.c: lxc_create_network_priv: 3427 No such file or directory - Failed to create network device
lxc-start: waydroid: ../src/lxc/start.c: lxc_spawn: 1840 Failed to create the network
lxc-start: waydroid: ../src/lxc/start.c: __lxc_start: 2107 Failed to spawn container "waydroid"
lxc-start: waydroid: ../src/lxc/conf.c: run_buffer: 321 Script exited with status 126
lxc-start: waydroid: ../src/lxc/start.c: lxc_end: 985 Failed to run lxc.hook.post-stop for container "waydroid"
lxc-start: waydroid: ../src/lxc/tools/lxc_start.c: main: 306 The container failed to start
lxc-start: waydroid: ../src/lxc/tools/lxc_start.c: main: 311 Additional information can be obtained by setting the --logfile and --logpriority options
(013323) [22:19:14] waiting 9 seconds for container to start...
(013323) [22:19:15] waiting 8 seconds for container to start...
(013323) [22:19:16] waiting 7 seconds for container to start...
(013323) [22:19:17] waiting 6 seconds for container to start...
(013323) [22:19:18] waiting 5 seconds for container to start...
(013323) [22:19:19] waiting 4 seconds for container to start...
(013323) [22:19:20] waiting 3 seconds for container to start...
(013323) [22:19:21] waiting 2 seconds for container to start...
(013323) [22:19:22] waiting 1 seconds for container to start...
(013323) [22:19:23] ERROR: container failed to start
(013323) [22:19:23] See also: <https://github.com/waydroid>
(013323) [22:19:23] Traceback (most recent call last):
  File "/usr/lib/waydroid/tools/__init__.py", line 68, in main
    actions.container_manager.start(args)
  File "/usr/lib/waydroid/tools/actions/container_manager.py", line 131, in start
    raise OSError("container failed to start")
OSError: container failed to start

(014031) [22:19:23] UserMonitor service is not even started
(014031) [22:19:23] Clipboard service is not even started
(013716) [22:19:23] UserMonitor service is not even started
(013716) [22:19:23] Clipboard service is not even started
(014032) [22:19:23] WayDroid container is STOPPED
(014043) [22:19:23] % modprobe -q ashmem_linux
(014043) [22:19:23] % chmod 666 -R /dev/anbox-binder
(014043) [22:19:23] % chmod 666 -R /dev/anbox-vndbinder
(014043) [22:19:23] % chmod 666 -R /dev/anbox-hwbinder
(014043) [22:19:23] Container manager is waiting for session to load

There are no messages in dmesg regarding the interface, or anything network-related.

The /run/waydroid-lxc directory is 777 permissioned and empty.

I have dnscrypt-proxy running on port 53, but it's complaining about a pidfile, not the TCP socket. I also tried to relaunch Waydroid container and session with dnscrypt-proxy stopped.

I tried to run dnsmasq standalone from a root shell, with the following command:

dnsmasq -x /run/waydroid-lxc/dnsmasq.pid

It didn't work, and failed with the same error message as is in the log. It started successfully when the pidfile was not specified (it was created at the default location /run/dnsmasq.pid).

notramo commented 1 year ago

I tried editing the pidfile option in /usr/lib/waydroid/data/scripts/waydroid-net.sh to a random one in /tmp, and it still did not work.

Then I completely deleted the pidfile option, and it successfully boots up the Android runtime, and waydroid app list gives meaningful result by listing actual apps.

notramo commented 1 year ago

However, there is no network access inside the Android system. I can't tell if it is a DNS configuration problem, or a routing problem, because I don't know how to test it. How can I get a shell without ADB? (Or how can I open ADB to Waydroid?)

JamiKettunen commented 1 year ago

waydroid shell gives you a root shell in the LXC container, I'll have to check later if I've ever had this dnsmasq.pid in my logs

JamiKettunen commented 1 year ago

As for the network issue it could be various things really including a running service such as docker causing the issue or a firewall setup, unfortunately https://docs.waydro.id/debugging/networking-issues doesn't really have too much info :/

notramo commented 1 year ago

I tried flushing nftables, no success. Also tried it with Docker stopped. ip route inside the container gives no output. ip addr show output inside the container:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0@if71: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:f9:d3:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::a758:6767:86f8:7586/64 scope link stable-privacy
       valid_lft forever preferred_lft forever

It seems it got an IPv6 address only, but the host system has only an IPv4 gateway.

ip addr show on the host system shows these 2 interfaces that are created by Waydroid:

    link/ether 00:16:3e:00:00:01 brd ff:ff:ff:ff:ff:ff
    inet 192.168.240.1/24 brd 192.168.240.255 scope global waydroid0
       valid_lft forever preferred_lft forever
    inet 169.254.106.76/16 brd 169.254.255.255 scope global noprefixroute waydroid0
       valid_lft forever preferred_lft forever
    inet6 fe80::ba14:a2bc:d8:3e6e/64 scope link
       valid_lft forever preferred_lft forever
71: vethvPyc2L@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master waydroid0 state UP group default qlen 1000
    link/ether fe:98:50:59:58:6a brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 169.254.59.163/16 brd 169.254.255.255 scope global noprefixroute vethvPyc2L
       valid_lft forever preferred_lft forever
    inet6 fe80::6625:f411:d258:7207/64 scope link
       valid_lft forever preferred_lft forever
notramo commented 1 year ago

I found the dnsmasq one. https://docs.waydro.id/debugging/known-issues

However, there is still no internet inside the container.

JamiKettunen commented 1 year ago

You mentioned you had dnscrypt-proxy running on port 53 and I found https://github.com/waydroid/waydroid/issues/117#issuecomment-950303832 as well as https://github.com/waydroid/waydroid/issues/588 which could be relevant.

Since you said you had docker running, if restarting waydroid-container service didn't help solve networking issue you maybe rebooting would? I previously filed https://github.com/waydroid/waydroid/issues/509 for the nftables + docker setup causing networking issues.

If you end up getting it working and have ideas on implementing the Waydroid network setup better do please contribute or file an issue upstream!

CameronNemo commented 1 year ago

Is AppArmor in use here? That could explain a dnsmasq permissions error.

JamiKettunen commented 1 year ago

Upstream actually does have AppArmor profiles too since v1.3.4 (https://github.com/waydroid/waydroid/tree/1.3.4/data/configs/apparmor_profiles), we just don't package them (yet); simply additionally calling make install_apparmor should get those in (https://github.com/waydroid/waydroid/blob/1.3.4/Makefile#L37), and I suppose it shouldn't need any extra dependencies as people who want to use AppArmor set it up themselves on Void

CameronNemo commented 1 year ago

I was actually referring to this, which it seems could still be an issue:

https://docs.waydro.id/debugging/known-issues#one-cause-for-this-issues-is-apparmor-the-fix-is-as-follows

If the maintainer of waydroid (you, @JamiKettunen?) can test with AppArmor that would be ideal, otherwise we should not install the Waydroid profiles ... I am not sure that they tested it! Maybe in a separate package for the atypical use case. Most people should understand that Waydroid provides a relatively medium-soft security boundary... still on the same kernel and hypervisor.

Edit: Made an MR for the apparmor profile change https://gitlab.com/apparmor/apparmor/-/merge_requests/969

HadetTheUndying commented 1 year ago

I'm also experiencing the issues of just no networking within the container. I've followed everything talked about here and still nothing works.