Emulator is stuck on the boot animation

[NoelOmo]

I have a running emulator that is stuck at the boot screen, Im stuck with an endles out put of the following log messages: 0000000002 kernel: [ 330.027687] RBP: ffff9a59c11df2a0 R08: 0000000000000003 R09: ffff89d07b003800 kernel: [ 330.052439] R10: ffff89d078eeb300 R11: ffff9a59c11df398 R12: ffff9a59c11df300 kernel: [ 330.052439] R13: ffff89d07a5e5800 R14: ffff9a59c11df201 R15: 0000000000000002 kernel: [ 330.142708] FS: 0000000000000000(0000) GS:ffff89d07fd80000(006b) knlGS:00000000f782c494 kernel: [ 330.152733] CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033 kernel: [ 330.152733] CR2: 00000000fff7b0fc CR3: 0000000133008000 CR4: 00000000000006a0 kernel: [ 330.213259] Call Trace: kernel: [ 330.213259] ? vp_notify+0x13/0x17 kernel: [ 330.213259] virtqueue_kick+0x22/0x35 kernel: [ 330.213259] virtnet_send_command+0xf3/0x165 kernel: [ 330.250870] virtnet_set_rx_mode+0xa9/0x364 kernel: [ 330.266688] ? __this_cpu_preempt_check+0x13/0x15 kernel: [ 330.286777] ? __local_bh_enable_ip+0x58/0x76 kernel: [ 330.286777] ? local_bh_enable+0x15/0x17 kernel: [ 330.295865] __dev_set_rx_mode+0x7c/0x81 kernel: [ 330.309102] __dev_open+0xdf/0x14a kernel: [ 330.313704] __dev_change_flags+0xad/0x1ce kernel: [ 330.313704] dev_change_flags+0x22/0x57 kernel: [ 330.351405] do_setlink+0x3b2/0xc4b kernel: [ 330.351405] rtnl_newlink+0x623/0x7e9 kernel: [ 330.388227] ? rtnl_newlink+0x131/0x7e9 kernel: [ 330.388227] ? constraint_expr_eval+0x230/0x4c6 kernel: [ 330.388227] ? selinux_capable+0x1e/0x20 kernel: [ 330.436923] ? security_capable+0x3d/0x56 kernel: [ 330.436923] ? ns_capable_common+0x2a/0x58 kernel: [ 330.477261] ? ns_capable+0xe/0x10 kernel: [ 330.477261] rtnetlink_rcv_msg+0x260/0x35a kernel: [ 330.485796] ? selinux_capable+0x1e/0x20 kernel: [ 330.495256] ? netdev_run_todo+0x5d/0x2dd kernel: [ 330.544774] ? ns_capable_common+0x2a/0x58 kernel: [ 330.544774] ? ns_capable+0xe/0x10 kernel: [ 330.561064] ? avc_has_perm+0x89/0x153 kernel: [ 330.561064] ? rtnetlink_bind+0x29/0x29 kernel: [ 330.595316] netlink_rcv_skb+0x7e/0xf7 kernel: [ 330.595316] rtnetlink_rcv+0x10/0x12 kernel: [ 330.602731] netlink_unicast+0x157/0x1e0 kernel: [ 330.602731] netlink_sendmsg+0x354/0x39b kernel: [ 330.633610] sock_sendmsg+0x2d/0x3d kernel: [ 330.633610] ___sys_sendmsg+0x1be/0x25c kernel: [ 330.633610] ? sock_destroy_inode+0x2e/0x31 kernel: [ 330.664827] ? evict+0x16f/0x184 kernel: [ 330.664827] ? __this_cpu_preempt_check+0x13/0x15 kernel: [ 330.686928] ? mem_cgroup_event_ratelimit+0x39/0x9c kernel: [ 330.693701] ? debug_smp_processor_id+0x17/0x19 kernel: [ 330.713638] ? __lru_cache_add+0x82/0x9a kernel: [ 330.713638] ? lru_cache_add_active_or_unevictable+0x49/0xba kernel: [ 330.766162] ? _raw_spin_unlock+0x15/0x2a kernel: [ 330.766162] __sys_sendmsg+0x4d/0x8a kernel: [ 330.766162] compat_SyS_sendmsg+0xf/0x11 kernel: [ 330.792805] compat_SyS_socketcall+0x1e8/0x249 kernel: [ 330.802261] ? page_fault+0x2f/0x50 kernel: [ 330.827916] do_fast_syscall_32+0x9e/0xf6 kernel: [ 330.827916] entry_SYSENTER_compat+0x7f/0x8e kernel: [ 330.827916] RIP: 0023:0xf797bad9 kernel: [ 330.843117] RSP: 002b:00000000fff7b0d0 EFLAGS: 00200286 ORIG_RAX: 0000000000000066 kernel: [ 330.875074] RAX: ffffffffffffffda RBX: 0000000000000010 RCX: 00000000fff7b0f0 kernel: [ 330.893801] RDX: 00000000fff83148 RSI: 0000000057296070 RDI: 00000000fff831e8 kernel: [ 330.907002] RBP: 00000000fff83168 R08: 0000000000000000 R09: 0000000000000000 kernel: [ 330.927084] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 kernel: [ 330.957693] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 kernel: [ 330.965052] Code: 01 00 01 00 72 07 0f b7 d6 89 f8 ee c3 55 48 89 e5 48 89 f7 48 c7 c6 89 f9 8b b7 e8 c8 fe ff ff 5d c3 48 81 fe 00 00 04 00 72 04 <66> 89 3e c3 48 81 fe 01 00 01 00 72 08 0f b7 d6 89 f8 66 ef c3 kernel: [ 333.170300] healthd: battery l=100 v=5000 t=25.0 h=2 st=2 c=900 fc=300000 cc=10 chg=a kernel: [ 357.883278] watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [ip:1835] kernel: [ 357.883278] Modules linked in: kernel: [ 357.903964] CPU: 3 PID: 1835 Comm: ip Tainted: G L 4.14.112+ #1 kernel: [ 357.915650] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014 kernel: [ 357.937638] task: 000000005c3e66ae task.stack: 000000005b83e08c kernel: [ 358.007255] RIP: 0010:iowrite16+0x9/0x33 kernel: [ 358.012751] RSP: 0018:ffff9a59c11df298 EFLAGS: 00210282 ORIG_RAX: ffffffffffffff10 kernel: [ 358.029704] RAX: 0000000000000002 RBX: ffff89d079a3b800 RCX: 0000000000000001 kernel: [ 358.051786] RDX: 00000000ffffffff RSI: ffff9a59c0145008 RDI: 0000000000000002 kernel: [ 358.051786] RBP: ffff9a59c11df2a0 R08: 0000000000000003 R09: ffff89d07b003800 kernel: [ 358.086856] R10: ffff89d078eeb300 R11: ffff9a59c11df398 R12: ffff9a59c11df300 kernel: [ 358.101066] R13: ffff89d07a5e5800 R14: ffff9a59c11df201 R15: 0000000000000002 kernel: [ 358.122691] FS: 0000000000000000(0000) GS:ffff89d07fd80000(006b) knlGS:00000000f782c494 kernel: [ 358.156319] CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033 kernel: [ 358.156319] CR2: 00000000fff7b0fc CR3: 0000000133008000 CR4: 00000000000006a0 kernel: [ 358.179137] Call Trace: kernel: [ 358.179137] ? vp_notify+0x13/0x17 kernel: [ 358.205771] virtqueue_kick+0x22/0x35 kernel: [ 358.205771] virtnet_send_command+0xf3/0x165 kernel: [ 358.205771] virtnet_set_rx_mode+0xa9/0x364 kernel: [ 358.205771] ? __this_cpu_preempt_check+0x13/0x15 kernel: [ 358.205771] ? __local_bh_enable_ip+0x58/0x76 kernel: [ 358.280505] ? local_bh_enable+0x15/0x17 kernel: [ 358.301796] __dev_set_rx_mode+0x7c/0x81 kernel: [ 358.314832] __dev_open+0xdf/0x14a kernel: [ 358.314832] __dev_change_flags+0xad/0x1ce kernel: [ 358.352652] dev_change_flags+0x22/0x57 kernel: [ 358.352652] do_setlink+0x3b2/0xc4b kernel: [ 358.370622] rtnl_newlink+0x623/0x7e9 kernel: [ 358.370622] ? rtnl_newlink+0x131/0x7e9 kernel: [ 358.414222] ? constraint_expr_eval+0x230/0x4c6 kernel: [ 358.414222] ? selinux_capable+0x1e/0x20 kernel: [ 358.435066] ? security_capable+0x3d/0x56 kernel: [ 358.441752] ? ns_capable_common+0x2a/0x58 kernel: [ 358.470925] ? ns_capable+0xe/0x10 kernel: [ 358.477068] rtnetlink_rcv_msg+0x260/0x35a kernel: [ 358.477068] ? selinux_capable+0x1e/0x20 kernel: [ 358.493701] ? netdev_run_todo+0x5d/0x2dd kernel: [ 358.516854] ? ns_capable_common+0x2a/0x58 kernel: [ 358.516854] ? ns_capable+0xe/0x10 kernel: [ 358.523680] ? avc_has_perm+0x89/0x153 kernel: [ 358.544447] ? rtnetlink_bind+0x29/0x29 kernel: [ 358.552203] netlink_rcv_skb+0x7e/0xf7 kernel: [ 358.552203] rtnetlink_rcv+0x10/0x12 kernel: [ 358.595722] netlink_unicast+0x157/0x1e0 kernel: [ 358.595722] netlink_sendmsg+0x354/0x39b kernel: [ 358.623663] sock_sendmsg+0x2d/0x3d kernel: [ 358.623663] ___sys_sendmsg+0x1be/0x25c kernel: [ 358.631179] ? sock_destroy_inode+0x2e/0x31 kernel: [ 358.631179] ? evict+0x16f/0x184 kernel: [ 358.652531] ? __this_cpu_preempt_check+0x13/0x15 kernel: [ 358.670673] ? mem_cgroup_event_ratelimit+0x39/0x9c kernel: [ 358.678706] ? debug_smp_processor_id+0x17/0x19 kernel: [ 358.683821] ? __lru_cache_add+0x82/0x9a kernel: [ 358.702399] ? lru_cache_add_active_or_unevictable+0x49/0xba kernel: [ 358.714877] ? _raw_spin_unlock+0x15/0x2a kernel: [ 358.714877] __sys_sendmsg+0x4d/0x8a kernel: [ 358.730951] compat_SyS_sendmsg+0xf/0x11 kernel: [ 358.730951] compat_SyS_socketcall+0x1e8/0x249 kernel: [ 358.763196] ? page_fault+0x2f/0x50 kernel: [ 358.763196] do_fast_syscall_32+0x9e/0xf6 kernel: [ 358.784814] entry_SYSENTER_compat+0x7f/0x8e kernel: [ 358.784814] RIP: 0023:0xf797bad9 kernel: [ 358.811531] RSP: 002b:00000000fff7b0d0 EFLAGS: 00200286 ORIG_RAX: 0000000000000066 kernel: [ 358.828066] RAX: ffffffffffffffda RBX: 0000000000000010 RCX: 00000000fff7b0f0 kernel: [ 358.828066] RDX: 00000000fff83148 RSI: 0000000057296070 RDI: 00000000fff831e8 kernel: [ 358.883942] RBP: 00000000fff83168 R08: 0000000000000000 R09: 0000000000000000 kernel: [

Anyone knows what this is?

[741g]

It looks like an issue in virtio-net. What API level are you seeing ths in?

[pokowaka]

Could you share the first log lines of the the docker container? (prefixed with version: tag). This will list the emulator version, and API level. Sharing uname -a might be useful for us as well.

[NoelOmo]

The following are the details for the emulator: tail: '/tmp/android-unknown/goldfish_rtc_0' has appeared; following new file version: Android emulator version 29.2.11.0 (build_id 6031357) (CL:N/A) version: launch_script: 0+unknown version: Pkg.Desc=System Image x86 with Google APIs. version: Pkg.Revision=9 version: Pkg.Dependencies=emulator#28.1.9 version: AndroidVersion.ApiLevel=29 version: SystemImage.Abi=x86 version: SystemImage.TagId=google_apis version: SystemImage.TagDisplay=Google APIs version: SystemImage.GpuSupport=true version: Addon.VendorId=google version: Addon.VendorDisplay=Google Inc.

The following are uname -a details: Linux XXXXXXXXXXXX 3.10.0-957.10.1.el7.x86_64 #1 SMP Mon Mar 18 15:06:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

[NoelOmo]

Just checked the docker health logs, it looks like the device is not even present: { "Status":"unhealthy", "FailingStreak":29, "Log":[ { "Start":"2019-12-06T10:27:02.0484238+03:00", "End":"2019-12-06T10:27:02.415746618+03:00", "ExitCode":1, "Output":"error: device 'emulator-6554' not found\n" }, { "Start":"2019-12-06T10:27:32.509415874+03:00", "End":"2019-12-06T10:27:32.929243945+03:00", "ExitCode":1, "Output":"error: device 'emulator-6554' not found\n" }, { "Start":"2019-12-06T10:28:03.013466443+03:00", "End":"2019-12-06T10:28:03.465189398+03:00", "ExitCode":1, "Output":"error: device 'emulator-6554' not found\n" }, { "Start":"2019-12-06T10:28:33.563481403+03:00", "End":"2019-12-06T10:28:34.002096161+03:00", "ExitCode":1, "Output":"error: device 'emulator-6554' not found\n" }, { "Start":"2019-12-06T10:29:04.080411393+03:00", "End":"2019-12-06T10:29:04.442702304+03:00", "ExitCode":1, "Output":"error: device 'emulator-6554' not found\n" } ] }

[pokowaka]

Let me see if I can re-create a rhel environment to reproduce your failure.

[NoelOmo]

Let me see if I can re-create a rhel environment to reproduce your failure.

Have you been able to reproduce this on your local @pokowaka

[NoelOmo]

Hey @pokowaka any updates on this issue?

[pokowaka]

I have not yet had a chance to setup a RHEL environment.

[dmarjanovic]

@NoelOmo did you have any luck with solving this? I'm seeing similar soft lockup as you with similar stack trace but on CentOS 7.8 in https://github.com/google/android-emulator-container-scripts/issues/171 :

kernel: [    8.177803] IPv6: ADDRCONF(NETDEV_UP): radio0-peer: link is not ready
kernel: [   32.490032] watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [ip:1810]
kernel: [   32.492694] Modules linked in:
kernel: [   32.494078] CPU: 0 PID: 1810 Comm: ip Not tainted 4.14.150+ #1
...
kernel: [   60.494065] CPU: 0 PID: 1810 Comm: ip Tainted: G             L  4.14.150+ #1
...

/cc @pokowaka