search

ssrg-vt / popcorn-kernel

Popcorn Linux kernel for distributed thread execution
Other
156 stars 22 forks source link

'stack' example triggers series of BUG statements on remote #77

Closed bxatnarf closed 4 years ago

bxatnarf commented 5 years ago

branch: merge arch: x86-64 triggering example: https://github.com/ssrg-vt/popcorn-kernel-lib/blob/master/src/stack.c

~NB: stack does not successfully execute on native popcorn (master branch) either (it segfaults), but no BUG statement are triggered in this version.~ NB: stack works for me if I compile it on one of the VMs. I'm guessing that it has something to do with its use of __builtin_frame_address. Therefore, to trigger this bug you should build stack on a different enough x86-64 host .

After stack exits (unsuccessfully), the remote kernel encounters bad state. Remote kernel log (truncated): notice that many of these stack traces say "BUG: Bad page state in process bash pfn:1374cd" There are also a few that reference the sshd process.

[   37.491970] remote_worker_main: [634] for [631/0]
[   37.491970] remote_worker_main: [634] /home/popcorn/popcorn-kernel-lib/src/stack
[   37.522549] remote_thread_main [635] started for [631/0]
[   37.522728] restore_thread_info [635] ip 40206f
[   37.523060] restore_thread_info [635] sp 7fffffffe220 bp 7fffffffe4d0
[   37.523447] restore_thread_info [635] fs 4cf880 fpu active
[   37.526250] 
[   37.526250] ####### MIGRATED - [635/1] from [631/0]
[   37.527634] 
[   37.527634] ## VMAFAULT [635] 40206f 40206f
[   37.530537]   [635] 402000 ->[631/0]
[   37.574707]   [635] 0 402000
[   37.576308]   [635] /home/popcorn/popcorn-kernel-lib/src/stack + 1
[   37.577641]   [635] map+ 401000 49d000
[   37.581534]   [635] map0 401000 -- 49d000 @ 1, 5
[   37.584268] 
[   37.584268] ## PAGEFAULT [635] 402000 R 40206f 154 0
[   37.588545]   [635] VM_EXEC. continue
[   37.591845] 
[   37.591845] ## PAGEFAULT [635] 402000 R 40206f 170 0
[   37.593535]   [635] VM_EXEC. continue
[   37.595167] 
[   37.595167] ## VMAFAULT [635] 7fffffffe288 402072
[   37.600407]   [635] 7fffffffe000 ->[631/0]
[   37.605990]   [635] 0 7fffffffe000
[   37.605990]   [635] map+ 7ffffffde000 7ffffffff000
[   37.606274]   [635] map0 7ffffffde000 -- 7ffffffff000 @ 7ffffffde, 3
[   37.606788] 
[   37.606788] ## PAGEFAULT [635] 7fffffffe000 R 402072 54 0
[   37.607917]  =[635] 7fffffffe000 00000000ef689557
[   37.609595]   [635] ->[631/0] 7fffffffe000 402072
[   37.622229]   [635] <-[631/0] 7fffffffe000 0
[   37.626603] >>[635] 7fffffffe000 00000000ef689557
[   37.627669] 
[   37.627669] ## PAGEFAULT [635] 7fffffffe000 W 402126 55 8000000000000025
[   37.629726]  =[635] 7fffffffe000 0000000069217a16
[   37.631468]   [635] ->[631/0] 7fffffffe000 402126
[   37.638182]   [635] <-[631/0] 7fffffffe000 4000
[   37.641175] >>[635] 7fffffffe000 0000000069217a16
[   37.643232] 
[   37.643232] ## VMAFAULT [635] 4cf8a8 4022c0
[   37.645707]   [635] 4cf000 ->[631/0]
[   37.654194]   [635] 7ffffffde000 4cf000
[   37.655345]   [635] map+ 4ca000 4f2000
[   37.656720]   [635] map0 4ca000 -- 4f2000 @ 4ca, 3
[   37.661619] 
[   37.661619] ## PAGEFAULT [635] 4cf000 R 4022c0 54 0
[   37.662043]  =[635] 4cf000 00000000ef689557
[   37.662272]   [635] ->[631/0] 4cf000 4022c0
[   37.665907]   [635] <-[631/0] 4cf000 0
[   37.668076] >>[635] 4cf000 00000000ef689557
[   37.668076] 
[   37.668076] ## PAGEFAULT [635] 4ce000 R 402349 54 0
[   37.669640]  =[635] 4ce000 00000000ef689557
[   37.670103]   [635] ->[631/0] 4ce000 402349
[   37.674168]   [635] <-[631/0] 4ce000 0
[   37.676199] >>[635] 4ce000 00000000ef689557
[   37.676199] 
[   37.676199] ## VMAFAULT [635] 49d004 4022ff
[   37.678896]   [635] 49d000 ->[631/0]
[   37.683935]   [635] 4ca000 49d000
[   37.685768]   [635] /home/popcorn/popcorn-kernel-lib/src/stack + 9d
[   37.687937]   [635] map+ 49d000 4c3000
[   37.689519]   [635] map0 49d000 -- 4c3000 @ 9d, 1
[   37.691303] 
[   37.691303] ## PAGEFAULT [635] 49d000 R 4022ff 14 0
[   37.693560]   [635] locally file-mapped read-only. continue
[   37.699605] 
[   37.699605] ## PAGEFAULT [635] 49d000 R 4022ff 30 0
[   37.701540]   [635] locally file-mapped read-only. continue
[   37.703807] 
[   37.703807] ## VMAFAULT [635] 4c7780 4097cd
[   37.706350]   [635] 4c7000 ->[631/0]
[   37.710277]   [635] 4ca000 4c7000
[   37.711736]   [635] /home/popcorn/popcorn-kernel-lib/src/stack + c3
[   37.713585]   [635] map+ 4c4000 4ca000
[   37.716401]   [635] map0 4c4000 -- 4ca000 @ c3, 3
[   37.716401] 
[   37.716401] ## PAGEFAULT [635] 4c7000 R 4097cd 54 0
[   37.721560]  =[635] 4c7000 0000000069217a16
[   37.723258]   [635] ->[631/0] 4c7000 4097cd
[   37.730680]   [635] <-[631/0] 4c7000 0
[   37.732092] >>[635] 4c7000 0000000069217a16
[   37.735518] 
[   37.735518] ## PAGEFAULT [635] 455000 R 455d50 154 0
[   37.737601]   [635] VM_EXEC. continue
[   37.743695] 
[   37.743695] ## PAGEFAULT [635] 455000 R 455d50 170 0
[   37.745564]   [635] VM_EXEC. continue
[   37.747011] 
[   37.747011] ## PAGEFAULT [635] 7fffffffd000 W 455d8a 55 0
[   37.749590]  =[635] 7fffffffd000 0000000069217a16
[   37.753613]   [635] ->[631/0] 7fffffffd000 455d8a
[   37.758826]   [635] <-[631/0] 7fffffffd000 0
[   37.762038] >>[635] 7fffffffd000 0000000069217a16
[   37.763908] 
[   37.763908] ## PAGEFAULT [635] 4c6000 R 455deb 54 0
[   37.765980]  =[635] 4c6000 0000000069217a16
[   37.767468]   [635] ->[631/0] 4c6000 455deb
[   37.774451]   [635] <-[631/0] 4c6000 0
[   37.774755] >>[635] 4c6000 0000000069217a16
[   37.775110] 
[   37.775110] ## PAGEFAULT [635] 434000 R 434ac0 154 0
[   37.775670]   [635] VM_EXEC. continue
[   37.779513] 
[   37.779513] ## PAGEFAULT [635] 4cd000 R 456013 54 0
[   37.780497]  =[635] 4cd000 0000000069217a16
[   37.781138]   [635] ->[631/0] 4cd000 456013
[   37.791521]   [635] <-[631/0] 4cd000 0
[   37.791521] >>[635] 4cd000 0000000069217a16
[   37.794378] 
[   37.794378] ## PAGEFAULT [635] 4cd000 W 456031 55 8000000000000025
[   37.794928]  =[635] 4cd000 0000000069217a16
[   37.795273]   [635] ->[631/0] 4cd000 456031
[   37.798843]   [635] <-[631/0] 4cd000 4000
[   37.799356] >>[635] 4cd000 0000000069217a16
[   37.799715] 
[   37.799715] ## PAGEFAULT [635] 4c9000 R 455e5f 54 0
[   37.800067]  =[635] 4c9000 0000000069217a16
[   37.800292]   [635] ->[631/0] 4c9000 455e5f
[   37.804627]   [635] <-[631/0] 4c9000 0
[   37.805910] >>[635] 4c9000 0000000069217a16
[   37.809648] 
[   37.809648] ## PAGEFAULT [635] 4ae000 R 455f33 54 0
[   37.812645]   [635] locally file-mapped read-only. continue
[   37.815416] 
[   37.815416] ## PAGEFAULT [635] 4ae000 R 455f33 70 0
[   37.817560]   [635] locally file-mapped read-only. continue
[   37.817959] 
[   37.817959] ## PAGEFAULT [635] 4c4000 R 455f52 54 0
[   37.818358]  =[635] 4c4000 0000000069217a16
[   37.818549]   [635] ->[631/0] 4c4000 455f52
[   37.822629]   [635] <-[631/0] 4c4000 0
[   37.825662] >>[635] 4c4000 0000000069217a16
[   37.826605] 
[   37.826605] ## PAGEFAULT [635] 478000 R 478bc0 154 0
[   37.826933]   [635] VM_EXEC. continue
[   37.829558] 
[   37.829558] ## PAGEFAULT [635] 478000 R 478bc0 170 0
[   37.829956]   [635] VM_EXEC. continue
[   37.830614] 
[   37.830614] ## PAGEFAULT [635] 4b4000 R 478c1d 54 0
[   37.830955]   [635] locally file-mapped read-only. continue
[   37.831558] 
[   37.831558] ## PAGEFAULT [635] 4c8000 R 40f220 54 0
[   37.831951]  =[635] 4c8000 0000000069217a16
[   37.832179]   [635] ->[631/0] 4c8000 40f220
[   37.835788]   [635] <-[631/0] 4c8000 0
[   37.837666] >>[635] 4c8000 0000000069217a16
[   37.841652] 
[   37.841652] ## PAGEFAULT [635] 4c7000 W 40e5d5 55 8000000000000025
[   37.845680]  =[635] 4c7000 0000000069217a16
[   37.845884]   [635] ->[631/0] 4c7000 40e5d5
[   37.853971]   [635] <-[631/0] 4c7000 4000
[   37.854857] >>[635] 4c7000 0000000069217a16
[   37.854857] 
[   37.854857] ## PAGEFAULT [635] 4d0000 W 40e5d9 55 0
[   37.854857]  =[635] 4d0000 0000000069217a16
[   37.854857]   [635] ->[631/0] 4d0000 40e5d9
[   37.859380]   [635] <-[631/0] 4d0000 0
[   37.860354] >>[635] 4d0000 0000000069217a16
[   37.860354] 
[   37.860354] ## PAGEFAULT [635] 43b000 R 43b6b0 154 0
[   37.861631]   [635] VM_EXEC. continue
[   37.862350] 
[   37.862350] ## PAGEFAULT [635] 4cf000 W 4021f9 55 8000000000000025
[   37.862777]  =[635] 4cf000 0000000069217a16
[   37.863082]   [635] ->[631/0] 4cf000 4021f9
[   37.868207]   [635] <-[631/0] 4cf000 4000
[   37.869337] >>[635] 4cf000 0000000069217a16
[   37.870577] traps: stack[635] general protection fault ip:401e38 sp:7fffffffe4c0 error]
[   37.872789] EXITED [635] remote / 0xb
[   37.879473] 
[   37.879473] TERMINATE [634] with 0xb
[   37.880206] EXITED [634] remote worker / 0xb
[   37.880756]   [634] zap 401000
[   37.881586]   [634] zap 402000
[   37.881787]   [634] zap 403000
[   37.882206]   [634] zap 404000
[   37.882490]   [634] zap 405000
[   37.882770]   [634] zap 406000
[   37.883165]   [634] zap 407000
[   37.883455]   [634] zap 408000
[   37.883649]   [634] zap 409000
[   37.883939]   [634] zap 40a000
[   37.884301]   [634] zap 40b000
[   37.884731]   [634] zap 40c000
[   37.885859]   [634] zap 40d000
[   37.886213]   [634] zap 40e000
[   37.886619]   [634] zap 40f000
[   37.887065]   [634] zap 410000
[   37.887356]   [634] zap 430000
[   37.887534]   [634] zap 431000
[   37.887716]   [634] zap 432000
[   37.887914]   [634] zap 433000
[   37.888092]   [634] zap 434000
[   37.888268]   [634] zap 435000
[   37.888443]   [634] zap 436000
[   37.888636]   [634] zap 437000
[   37.888829]   [634] zap 438000
[   37.889005]   [634] zap 439000
[   37.889549]   [634] zap 43a000
[   37.889757]   [634] zap 43b000
[   37.890072]   [634] zap 43d000
[   37.890273]   [634] zap 43e000
[   37.890469]   [634] zap 43f000
[   37.890676]   [634] zap 450000
[   37.890903]   [634] zap 451000
[   37.891152]   [634] zap 452000
[   37.891341]   [634] zap 453000
[   37.891527]   [634] zap 454000
[   37.891719]   [634] zap 455000
[   37.891937]   [634] zap 456000
[   37.892147]   [634] zap 457000
[   37.892336]   [634] zap 458000
[   37.892536]   [634] zap 459000
[   37.892759]   [634] zap 45a000
[   37.892963]   [634] zap 45b000
[   37.893153]   [634] zap 45c000
[   37.893531]   [634] zap 45e000
[   37.893770]   [634] zap 45f000
[   37.893997]   [634] zap 470000
[   37.894271]   [634] zap 471000
[   37.894516]   [634] zap 472000
[   37.894715]   [634] zap 473000
[   37.894914]   [634] zap 474000
[   37.895095]   [634] zap 475000
[   37.895285]   [634] zap 476000
[   37.895484]   [634] zap 477000
[   37.895698]   [634] zap 478000
[   37.896074]   [634] zap 479000
[   37.896383]   [634] zap 47a000
[   37.896586]   [634] zap 47b000
[   37.896821]   [634] zap 47c000
[   37.897038]   [634] zap 47d000
[   37.897509]   [634] zap 47e000
[   37.897738]   [634] zap 47f000
[   37.898921]   [634] zap 49d000
[   37.899162]   [634] zap 49e000
[   37.899363]   [634] zap 49f000
[   37.899558]   [634] zap 4a0000
[   37.899750]   [634] zap 4a1000
[   37.899953]   [634] zap 4a2000
[   37.900148]   [634] zap 4a3000
[   37.900386]   [634] zap 4a4000
[   37.900578]   [634] zap 4a6000
[   37.900867]   [634] zap 4a7000
[   37.901130]   [634] zap 4a8000
[   37.901574]   [634] zap 4a9000
[   37.901905]   [634] zap 4aa000
[   37.902110]   [634] zap 4ab000
[   37.902283]   [634] zap 4ac000
[   37.902577]   [634] zap 4ad000
[   37.902796]   [634] zap 4ae000
[   37.902998]   [634] zap 4af000
[   37.903191]   [634] zap 4b0000
[   37.903379]   [634] zap 4b1000
[   37.903640]   [634] zap 4b2000
[   37.903941]   [634] zap 4b3000
[   37.904148]   [634] zap 4b4000
[   37.904435]   [634] zap 4b5000
[   37.904711]   [634] zap 4b7000
[   37.904965]   [634] zap 4b8000
[   37.905607]   [634] zap 4b9000
[   37.905852]   [634] zap 4ba000
[   37.906142]   [634] zap 4bb000
[   37.906336]   [634] zap 4bc000
[   37.906575]   [634] zap 4bd000
[   37.908019]   [634] zap 4c4000
[   37.908304]   [634] zap 4c6000
[   37.908525]   [634] zap 4c7000
[   37.908796]   [634] zap 4c8000
[   37.909057]   [634] zap 4c9000
[   37.910067]   [634] zap 4cd000
[   37.910356]   [634] zap 4ce000
[   37.910538]   [634] zap 4cf000
[   37.910800]   [634] zap 4d0000
[   37.911050]   [634] zap 7fffffffd000
[   37.911257]   [634] zap 7fffffffe000
[   52.516983] WARNING: CPU: 0 PID: 51 at mm/filemap.c:899 add_to_page_cache_lru+0xb1/0xc0
[   52.517544] Modules linked in: msg_socket
[   52.518007] CPU: 0 PID: 51 Comm: jbd2/sda1-8 Not tainted 4.20.0-rc7-popcorn+ #114
[   52.518359] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181124
[   52.518359] RIP: 0010:add_to_page_cache_lru+0xb1/0xc0
[   52.518359] Code: 2d 48 83 c4 10 5b 5d 41 5c c3 48 8b 34 24 48 85 f6 74 d4 48 89 df e8b
[   52.518359] RSP: 0018:ffffc900001e7c20 EFLAGS: 00000202
[   52.518359] RAX: 8000000000020037 RBX: ffffea00043e8100 RCX: 0000000000000000
[   52.518359] RDX: ffffea00043e8100 RSI: 00000000ffffffff RDI: ffffffff81c4dea0
[   52.518359] RBP: 0000000000000000 R08: ffffffffffe5a4a3 R09: 0000000000000001
[   52.518359] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000620848
[   52.518359] R13: ffff88813b04ea70 R14: 00000000000356b5 R15: ffffffff81c6ba60
[   52.518359] FS:  0000000000000000(0000) GS:ffff88813fa00000(0000) knlGS:000000000000000
[   52.518359] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   52.518359] CR2: 00007ffff5f1afc0 CR3: 0000000137964000 CR4: 00000000000006f0
[   52.518359] Call Trace:
[   52.518359]  pagecache_get_page+0x122/0x230
[   52.518359]  __getblk_gfp+0xfa/0x260
[   52.518359]  jbd2_journal_get_descriptor_buffer+0x56/0xf0
[   52.518359]  jbd2_journal_commit_transaction+0xd2f/0x1d19
[   52.518359]  ? try_to_del_timer_sync+0x4d/0x80
[   52.518359]  kjournald2+0xc9/0x2b0
[   52.518359]  ? wait_woken+0xa0/0xa0
[   52.518359]  ? commit_timeout+0x10/0x10
[   52.518359]  kthread+0x120/0x140
[   52.518359]  ? kthread_park+0x80/0x80
[   52.518359]  ret_from_fork+0x3a/0x50
[   52.518359] irq event stamp: 1292
[   52.518359] hardirqs last  enabled at (1291): [<ffffffff8152a0a9>] _raw_spin_unlock_ir0
[   52.518359] hardirqs last disabled at (1292): [<ffffffff810019d9>] trace_hardirqs_off_c
[   52.518359] softirqs last  enabled at (1022): [<ffffffff81800341>] __do_softirq+0x341/c
[   52.518359] softirqs last disabled at (1007): [<ffffffff81065276>] irq_exit+0xa6/0xe0
[   52.518359] ---[ end trace 0b218bc2feba34b8 ]---
[  389.019006] BUG: Bad page state in process sshd  pfn:135e5a
[  389.019333] page:ffffea00043ca3b0 count:1 mapcount:1 mapping:ffff88813a72f709 index:0x8
[  389.019389] flags: 0x8000000000080034(uptodate|lru|active|swapbacked)
[  389.019389] raw: 8000000000080034 dead000000000100 dead000000000200 ffff88813a72f709
[  389.019389] raw: 00000007ffff7fe8 0000000000000000 0000000100000000
[  389.019389] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  389.019389] bad because of flags: 0x30(lru|active)
[  389.019389] Modules linked in: msg_socket
[  389.019389] CPU: 0 PID: 589 Comm: sshd Tainted: G        W         4.20.0-rc7-popcorn+4
[  389.019389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181124
[  389.019389] Call Trace:
[  389.019389]  dump_stack+0x67/0x90
[  389.019389]  bad_page.cold.36+0x8f/0xc2
[  389.019389]  free_pcppages_bulk+0x18f/0x560
[  389.019389]  free_unref_page_list+0x116/0x240
[  389.019389]  release_pages+0x296/0x2e0
[  389.019389]  tlb_flush_mmu_free+0x31/0x50
[  389.019389]  arch_tlb_finish_mmu+0x2f/0xb0
[  389.019389]  tlb_finish_mmu+0x1f/0x30
[  389.019389]  exit_mmap+0xb1/0x160
[  389.019389]  mmput+0x41/0xf0
[  389.019389]  do_exit+0x38b/0xb50
[  389.019389]  ? _raw_spin_unlock_irq+0x29/0x40
[  389.019389]  do_group_exit+0x39/0xb0
[  389.019389]  __x64_sys_exit_group+0x14/0x20
[  389.019389]  do_syscall_64+0x69/0x400
[  389.019389]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  389.019389]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  389.019389] RIP: 0033:0x7ffff63362e9
[  389.019389] Code: Bad RIP value.
[  389.019389] RSP: 002b:00007fffffffe1d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  389.019389] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007ffff63362e9
[  389.019389] RDX: 00000000000000ff RSI: 0000000000000001 RDI: 00000000000000ff
[  389.019389] RBP: 0000555555813280 R08: 000000000000003c R09: 00000000000000e7
[  389.019389] R10: fffffffffffffe90 R11: 0000000000000246 R12: 00007fffffffe230
[  389.019389] R13: 0000000000000000 R14: 00005555558231d0 R15: 0000555555815e80
[  389.019389] Disabling lock debugging due to kernel taint
[  389.019389] BUG: Bad page state in process sshd  pfn:137279
[  389.019389] page:ffffea0004410a78 count:1 mapcount:1 mapping:ffff88813a74c4b1 index:0xe
[  389.019389] flags: 0x8000000000080034(uptodate|lru|active|swapbacked)
[  389.019389] raw: 8000000000080034 dead000000000100 dead000000000200 ffff88813a74c4b1
[  389.019389] raw: 00000007ffffffbe 0000000000000000 0000000100000000
[  389.019389] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  389.019389] bad because of flags: 0x30(lru|active)
[  389.019389] Modules linked in: msg_socket
[  389.019389] CPU: 0 PID: 589 Comm: sshd Tainted: G    B   W         4.20.0-rc7-popcorn+4
[  389.019389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181124
[  389.019389] Call Trace:
[  389.019389]  dump_stack+0x67/0x90
[  389.019389]  bad_page.cold.36+0x8f/0xc2
[  389.019389]  free_pcppages_bulk+0x18f/0x560
[  389.019389]  free_unref_page_list+0x116/0x240
[  389.019389]  release_pages+0x296/0x2e0
[  389.019389]  tlb_flush_mmu_free+0x31/0x50
[  389.019389]  arch_tlb_finish_mmu+0x2f/0xb0
[  389.019389]  tlb_finish_mmu+0x1f/0x30
[  389.019389]  exit_mmap+0xb1/0x160
[  389.019389]  mmput+0x41/0xf0
[  389.019389]  do_exit+0x38b/0xb50
[  389.019389]  ? _raw_spin_unlock_irq+0x29/0x40
[  389.019389]  do_group_exit+0x39/0xb0
[  389.019389]  __x64_sys_exit_group+0x14/0x20
[  389.019389]  do_syscall_64+0x69/0x400
[  389.019389]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  389.019389]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  389.019389] RIP: 0033:0x7ffff63362e9
[  389.019389] Code: Bad RIP value.
[  389.019389] RSP: 002b:00007fffffffe1d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  389.019389] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007ffff63362e9
[  389.019389] RDX: 00000000000000ff RSI: 0000000000000001 RDI: 00000000000000ff
[  389.019389] RBP: 0000555555813280 R08: 000000000000003c R09: 00000000000000e7
[  389.019389] R10: fffffffffffffe90 R11: 0000000000000246 R12: 00007fffffffe230
[  389.019389] R13: 0000000000000000 R14: 00005555558231d0 R15: 0000555555815e80
[  389.019389] BUG: Bad page state in process sshd  pfn:135e1a
[  389.019389] page:ffffea00043c95b0 count:1 mapcount:1 mapping:ffff888139ba30c9 index:0xf
[  389.019389] flags: 0x8000000000082034(uptodate|lru|active|private|swapbacked)
[  389.019389] raw: 8000000000082034 dead000000000100 dead000000000200 ffff888139ba30c9
[  389.019389] raw: 00000007ffff691f 0000000000000000 0000000100000000
[  389.019389] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  389.019389] bad because of flags: 0x2030(lru|active|private)
[  389.019389] Modules linked in: msg_socket
[  389.019389] CPU: 0 PID: 589 Comm: sshd Tainted: G    B   W         4.20.0-rc7-popcorn+4
[  389.019389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181124
[  389.019389] Call Trace:
[  389.019389]  dump_stack+0x67/0x90
[  389.019389]  bad_page.cold.36+0x8f/0xc2
[  389.019389]  free_pcppages_bulk+0x18f/0x560
[  389.019389]  free_unref_page_list+0x116/0x240
[  389.019389]  release_pages+0x296/0x2e0
[  389.019389]  tlb_flush_mmu_free+0x31/0x50
[  389.019389]  arch_tlb_finish_mmu+0x2f/0xb0
[  389.019389]  tlb_finish_mmu+0x1f/0x30
[  389.019389]  exit_mmap+0xb1/0x160
[  389.019389]  mmput+0x41/0xf0
[  389.019389]  do_exit+0x38b/0xb50
[  389.019389]  ? _raw_spin_unlock_irq+0x29/0x40
[  389.019389]  do_group_exit+0x39/0xb0
[  389.019389]  __x64_sys_exit_group+0x14/0x20
[  389.019389]  do_syscall_64+0x69/0x400
[  389.019389]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  389.019389]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  389.019389] RIP: 0033:0x7ffff63362e9
[  389.019389] Code: Bad RIP value.
[  389.019389] RSP: 002b:00007fffffffe1d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  389.019389] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007ffff63362e9
[  389.019389] RDX: 00000000000000ff RSI: 0000000000000001 RDI: 00000000000000ff
[  389.019389] RBP: 0000555555813280 R08: 000000000000003c R09: 00000000000000e7
[  389.019389] R10: fffffffffffffe90 R11: 0000000000000246 R12: 00007fffffffe230
[  389.019389] R13: 0000000000000000 R14: 00005555558231d0 R15: 0000555555815e80
[  389.019389] BUG: Bad page state in process sshd  pfn:135d50
[  389.019389] page:ffffea00043c6980 count:1 mapcount:1 mapping:ffff888139ba30c9 index:0x0
[  389.019389] flags: 0x8000000000082034(uptodate|lru|active|private|swapbacked)
[  389.019389] raw: 8000000000082034 dead000000000100 dead000000000200 ffff888139ba30c9
[  389.019389] raw: 00000007ffff6920 0000000000000000 0000000100000000
[  389.019389] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  389.019389] bad because of flags: 0x2030(lru|active|private)
[  389.019389] Modules linked in: msg_socket
[  389.019389] CPU: 0 PID: 589 Comm: sshd Tainted: G    B   W         4.20.0-rc7-popcorn+4
[  389.019389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181124
[  389.019389] Call Trace:
[  389.019389]  dump_stack+0x67/0x90
[  389.019389]  bad_page.cold.36+0x8f/0xc2
[  389.019389]  free_pcppages_bulk+0x18f/0x560
[  389.019389]  free_unref_page_list+0x116/0x240
[  389.019389]  release_pages+0x296/0x2e0
[  389.019389]  tlb_flush_mmu_free+0x31/0x50
[  389.019389]  arch_tlb_finish_mmu+0x2f/0xb0
[  389.019389]  tlb_finish_mmu+0x1f/0x30
[  389.019389]  exit_mmap+0xb1/0x160
[  389.019389]  mmput+0x41/0xf0
[  389.019389]  do_exit+0x38b/0xb50
[  389.019389]  ? _raw_spin_unlock_irq+0x29/0x40
[  389.019389]  do_group_exit+0x39/0xb0
[  389.019389]  __x64_sys_exit_group+0x14/0x20
[  389.019389]  do_syscall_64+0x69/0x400
[  389.019389]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  389.019389]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  389.019389] RIP: 0033:0x7ffff63362e9
[  389.019389] Code: Bad RIP value.
[  389.019389] RSP: 002b:00007fffffffe1d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  389.019389] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007ffff63362e9
[  389.019389] RDX: 00000000000000ff RSI: 0000000000000001 RDI: 00000000000000ff
[  389.019389] RBP: 0000555555813280 R08: 000000000000003c R09: 00000000000000e7
[  389.019389] R10: fffffffffffffe90 R11: 0000000000000246 R12: 00007fffffffe230
[  389.019389] R13: 0000000000000000 R14: 00005555558231d0 R15: 0000555555815e80
[  389.019389] BUG: Bad page state in process sshd  pfn:136789
[  389.019389] page:ffffea00043ea5f8 count:1 mapcount:1 mapping:ffff888139ba3d49 index:0x5
[  389.019389] flags: 0x8000000000082034(uptodate|lru|active|private|swapbacked)
[  389.019389] raw: 8000000000082034 dead000000000100 dead000000000200 ffff888139ba3d49
[  389.019389] raw: 00000000000001a5 0000000000000000 0000000100000000
[  389.019389] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  389.019389] bad because of flags: 0x2030(lru|active|private)
[  389.019389] Modules linked in: msg_socket
[  389.019389] CPU: 0 PID: 589 Comm: sshd Tainted: G    B   W         4.20.0-rc7-popcorn+4
[  389.019389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181124
[  389.019389] Call Trace:
[  389.019389]  dump_stack+0x67/0x90
[  389.019389]  bad_page.cold.36+0x8f/0xc2
[  389.019389]  free_pcppages_bulk+0x18f/0x560
[  389.019389]  free_unref_page_list+0x116/0x240
[  389.019389]  release_pages+0x296/0x2e0
[  389.019389]  tlb_flush_mmu_free+0x31/0x50
[  389.019389]  arch_tlb_finish_mmu+0x2f/0xb0
[  389.019389]  tlb_finish_mmu+0x1f/0x30
[  389.019389]  exit_mmap+0xb1/0x160
[  389.019389]  mmput+0x41/0xf0
[  389.019389]  do_exit+0x38b/0xb50
[  389.019389]  ? _raw_spin_unlock_irq+0x29/0x40
[  389.019389]  do_group_exit+0x39/0xb0
[  389.019389]  __x64_sys_exit_group+0x14/0x20
[  389.019389]  do_syscall_64+0x69/0x400
[  389.019389]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  389.019389]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  389.019389] RIP: 0033:0x7ffff63362e9
[  389.019389] Code: Bad RIP value.
[  389.019389] RSP: 002b:00007fffffffe1d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  389.019389] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007ffff63362e9
[  389.019389] RDX: 00000000000000ff RSI: 0000000000000001 RDI: 00000000000000ff
[  389.019389] RBP: 0000555555813280 R08: 000000000000003c R09: 00000000000000e7
[  389.019389] R10: fffffffffffffe90 R11: 0000000000000246 R12: 00007fffffffe230
[  389.019389] R13: 0000000000000000 R14: 00005555558231d0 R15: 0000555555815e80
[  389.019389] BUG: Bad page state in process sshd  pfn:135e5c
[  389.019389] page:ffffea00043ca420 count:2 mapcount:2 mapping:ffff888139aaf641 index:0xf
[  389.019389] flags: 0x8000000000082034(uptodate|lru|active|private|swapbacked)
[  389.019389] raw: 8000000000082034 dead000000000100 dead000000000200 ffff888139aaf641
[  389.019389] raw: 000000055555587f 0000000000000000 0000000200000001
[  389.019389] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  389.019389] bad because of flags: 0x2030(lru|active|private)
[  389.019389] Modules linked in: msg_socket
[  389.019389] CPU: 0 PID: 589 Comm: sshd Tainted: G    B   W         4.20.0-rc7-popcorn+4
[  389.019389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181124
[  389.019389] Call Trace:
[  389.019389]  dump_stack+0x67/0x90
[  389.019389]  bad_page.cold.36+0x8f/0xc2
[  389.019389]  free_pcppages_bulk+0x18f/0x560
[  389.019389]  free_unref_page_list+0x116/0x240
[  389.019389]  release_pages+0x296/0x2e0
[  389.019389]  tlb_flush_mmu_free+0x31/0x50
[  389.019389]  arch_tlb_finish_mmu+0x2f/0xb0
[  389.019389]  tlb_finish_mmu+0x1f/0x30
[  389.019389]  exit_mmap+0xb1/0x160
[  389.019389]  mmput+0x41/0xf0
[  389.019389]  do_exit+0x38b/0xb50
[  389.019389]  ? _raw_spin_unlock_irq+0x29/0x40
[  389.019389]  do_group_exit+0x39/0xb0
[  389.019389]  __x64_sys_exit_group+0x14/0x20
[  389.019389]  do_syscall_64+0x69/0x400
[  389.019389]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  389.019389]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  389.019389] RIP: 0033:0x7ffff63362e9
[  389.019389] Code: Bad RIP value.
[  389.019389] RSP: 002b:00007fffffffe1d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  389.019389] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007ffff63362e9
[  389.019389] RDX: 00000000000000ff RSI: 0000000000000001 RDI: 00000000000000ff
[  389.019389] RBP: 0000555555813280 R08: 000000000000003c R09: 00000000000000e7
[  389.019389] R10: fffffffffffffe90 R11: 0000000000000246 R12: 00007fffffffe230
[  389.019389] R13: 0000000000000000 R14: 00005555558231d0 R15: 0000555555815e80
[  389.019389] BUG: Bad page state in process sshd  pfn:135e5b
[  389.019389] page:ffffea00043ca3e8 count:2 mapcount:2 mapping:ffff888139aaf641 index:0xe
[  389.019389] flags: 0x8000000000082034(uptodate|lru|active|private|swapbacked)
[  389.019389] raw: 8000000000082034 dead000000000100 dead000000000200 ffff888139aaf641
[  389.019389] raw: 000000055555587e 0000000000000000 0000000200000001
[  389.019389] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  389.019389] bad because of flags: 0x2030(lru|active|private)
[  389.019389] Modules linked in: msg_socket
[  389.019389] CPU: 0 PID: 589 Comm: sshd Tainted: G    B   W         4.20.0-rc7-popcorn+4
[  389.019389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181124
[  389.019389] Call Trace:
[  389.019389]  dump_stack+0x67/0x90
[  389.019389]  bad_page.cold.36+0x8f/0xc2
[  389.019389]  free_pcppages_bulk+0x18f/0x560
[  389.019389]  free_unref_page_list+0x116/0x240
[  389.019389]  release_pages+0x296/0x2e0
[  389.019389]  tlb_flush_mmu_free+0x31/0x50
[  389.019389]  arch_tlb_finish_mmu+0x2f/0xb0
[  389.019389]  tlb_finish_mmu+0x1f/0x30
[  389.019389]  exit_mmap+0xb1/0x160
[  389.019389]  mmput+0x41/0xf0
[  389.019389]  do_exit+0x38b/0xb50
[  389.019389]  ? _raw_spin_unlock_irq+0x29/0x40
[  389.019389]  do_group_exit+0x39/0xb0
[  389.019389]  __x64_sys_exit_group+0x14/0x20
[  389.019389]  do_syscall_64+0x69/0x400
[  389.019389]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  389.019389]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  389.019389] RIP: 0033:0x7ffff63362e9
[  389.019389] Code: Bad RIP value.
[  389.019389] RSP: 002b:00007fffffffe1d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  389.019389] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007ffff63362e9
[  389.019389] RDX: 00000000000000ff RSI: 0000000000000001 RDI: 00000000000000ff
[  389.019389] RBP: 0000555555813280 R08: 000000000000003c R09: 00000000000000e7
[  389.019389] R10: fffffffffffffe90 R11: 0000000000000246 R12: 00007fffffffe230
[  389.019389] R13: 0000000000000000 R14: 00005555558231d0 R15: 0000555555815e80
[  389.019389] BUG: Bad page state in process sshd  pfn:135ddf
[  389.019389] page:ffffea00043c88c8 count:1 mapcount:1 mapping:ffff888139aaf641 index:0x1
[  389.019389] flags: 0x8000000000082034(uptodate|lru|active|private|swapbacked)
[  389.019389] raw: 8000000000082034 dead000000000100 dead000000000200 ffff888139aaf641
[  389.019389] raw: 0000000555555881 0000000000000000 0000000100000000
[  389.019389] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  389.019389] bad because of flags: 0x2030(lru|active|private)
[  389.019389] Modules linked in: msg_socket
[  389.019389] CPU: 0 PID: 589 Comm: sshd Tainted: G    B   W         4.20.0-rc7-popcorn+4
[  389.019389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181124
[  389.019389] Call Trace:
[  389.019389]  dump_stack+0x67/0x90
[  389.019389]  bad_page.cold.36+0x8f/0xc2
[  389.019389]  free_pcppages_bulk+0x18f/0x560
[  389.019389]  free_unref_page_list+0x116/0x240
[  389.019389]  release_pages+0x296/0x2e0
[  389.019389]  tlb_flush_mmu_free+0x31/0x50
[  389.019389]  arch_tlb_finish_mmu+0x2f/0xb0
[  389.019389]  tlb_finish_mmu+0x1f/0x30
[  389.019389]  exit_mmap+0xb1/0x160
[  389.019389]  mmput+0x41/0xf0
[  389.019389]  do_exit+0x38b/0xb50
[  389.019389]  ? _raw_spin_unlock_irq+0x29/0x40
[  389.019389]  do_group_exit+0x39/0xb0
[  389.019389]  __x64_sys_exit_group+0x14/0x20
[  389.019389]  do_syscall_64+0x69/0x400
[  389.019389]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  389.019389]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  389.019389] RIP: 0033:0x7ffff63362e9
[  389.019389] Code: Bad RIP value.
[  389.019389] RSP: 002b:00007fffffffe1d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  389.019389] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007ffff63362e9
[  389.019389] RDX: 00000000000000ff RSI: 0000000000000001 RDI: 00000000000000ff
[  389.019389] RBP: 0000555555813280 R08: 000000000000003c R09: 00000000000000e7
[  389.019389] R10: fffffffffffffe90 R11: 0000000000000246 R12: 00007fffffffe230
[  389.019389] R13: 0000000000000000 R14: 00005555558231d0 R15: 0000555555815e80
[  389.019389] BUG: Bad page state in process sshd  pfn:135dde
[  389.019389] page:ffffea00043c8890 count:2 mapcount:2 mapping:ffff888139aaf641 index:0x0
[  389.019389] flags: 0x8000000000082034(uptodate|lru|active|private|swapbacked)
[  389.019389] raw: 8000000000082034 dead000000000100 dead000000000200 ffff888139aaf641
[  389.019389] raw: 0000000555555880 0000000000000000 0000000200000001
[  389.019389] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  389.019389] bad because of flags: 0x2030(lru|active|private)
[  389.019389] Modules linked in: msg_socket
[  389.019389] CPU: 0 PID: 589 Comm: sshd Tainted: G    B   W         4.20.0-rc7-popcorn+4
[  389.019389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181124
[  389.019389] Call Trace:
[  389.019389]  dump_stack+0x67/0x90
[  389.019389]  bad_page.cold.36+0x8f/0xc2
[  389.019389]  free_pcppages_bulk+0x18f/0x560
[  389.019389]  free_unref_page_list+0x116/0x240
[  389.019389]  release_pages+0x296/0x2e0
[  389.019389]  tlb_flush_mmu_free+0x31/0x50
[  389.019389]  arch_tlb_finish_mmu+0x2f/0xb0
[  389.019389]  tlb_finish_mmu+0x1f/0x30
[  389.019389]  exit_mmap+0xb1/0x160
[  389.019389]  mmput+0x41/0xf0
[  389.019389]  do_exit+0x38b/0xb50
[  389.019389]  ? _raw_spin_unlock_irq+0x29/0x40
[  389.019389]  do_group_exit+0x39/0xb0
[  389.019389]  __x64_sys_exit_group+0x14/0x20
[  389.019389]  do_syscall_64+0x69/0x400
[  389.019389]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  389.019389]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  389.019389] RIP: 0033:0x7ffff63362e9
[  389.019389] Code: Bad RIP value.
[  389.019389] RSP: 002b:00007fffffffe1d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  389.019389] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007ffff63362e9
[  389.019389] RDX: 00000000000000ff RSI: 0000000000000001 RDI: 00000000000000ff
[  389.019389] RBP: 0000555555813280 R08: 000000000000003c R09: 00000000000000e7
[  389.019389] R10: fffffffffffffe90 R11: 0000000000000246 R12: 00007fffffffe230
[  389.019389] R13: 0000000000000000 R14: 00005555558231d0 R15: 0000555555815e80
[  389.019389] BUG: Bad page state in process sshd  pfn:1365ef
[  389.019389] page:ffffea00043e4c48 count:2 mapcount:2 mapping:ffff888139aaf641 index:0x3
[  389.019389] flags: 0x8000000000080034(uptodate|lru|active|swapbacked)
[  389.019389] raw: 8000000000080034 dead000000000100 dead000000000200 ffff888139aaf641
[  389.019389] raw: 0000000555555883 0000000000000000 0000000200000001
[  389.019389] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  389.019389] bad because of flags: 0x30(lru|active)
[  389.019389] Modules linked in: msg_socket
[  389.019389] CPU: 0 PID: 589 Comm: sshd Tainted: G    B   W         4.20.0-rc7-popcorn+4
[  389.019389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181124
[  389.019389] Call Trace:
[  389.019389]  dump_stack+0x67/0x90
[  389.019389]  bad_page.cold.36+0x8f/0xc2
[  389.019389]  free_pcppages_bulk+0x18f/0x560
[  389.019389]  free_unref_page_list+0x116/0x240
[  389.019389]  release_pages+0x296/0x2e0
[  389.019389]  tlb_flush_mmu_free+0x31/0x50
[  389.019389]  arch_tlb_finish_mmu+0x2f/0xb0
[  389.019389]  tlb_finish_mmu+0x1f/0x30
[  389.019389]  exit_mmap+0xb1/0x160
[  389.019389]  mmput+0x41/0xf0
[  389.019389]  do_exit+0x38b/0xb50
[  389.019389]  ? _raw_spin_unlock_irq+0x29/0x40
[  389.019389]  do_group_exit+0x39/0xb0
[  389.019389]  __x64_sys_exit_group+0x14/0x20
[  389.019389]  do_syscall_64+0x69/0x400
[  389.019389]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  389.019389]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  389.019389] RIP: 0033:0x7ffff63362e9
[  389.019389] Code: Bad RIP value.
[  389.019389] RSP: 002b:00007fffffffe1d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  389.019389] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007ffff63362e9
[  389.019389] RDX: 00000000000000ff RSI: 0000000000000001 RDI: 00000000000000ff
[  389.019389] RBP: 0000555555813280 R08: 000000000000003c R09: 00000000000000e7
[  389.019389] R10: fffffffffffffe90 R11: 0000000000000246 R12: 00007fffffffe230
[  389.019389] R13: 0000000000000000 R14: 00005555558231d0 R15: 0000555555815e80
[  389.019389] BUG: Bad page state in process sshd  pfn:135ddd
[  389.019389] page:ffffea00043c8858 count:1 mapcount:1 mapping:ffff888139aaf641 index:0x4
[  389.019389] flags: 0x8000000000082034(uptodate|lru|active|private|swapbacked)
[  389.019389] raw: 8000000000082034 dead000000000100 dead000000000200 ffff888139aaf641
[  389.019389] raw: 0000000555555884 0000000000000000 0000000100000000
[  389.019389] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  389.019389] bad because of flags: 0x2030(lru|active|private)
[  389.019389] Modules linked in: msg_socket
[  389.019389] CPU: 0 PID: 589 Comm: sshd Tainted: G    B   W         4.20.0-rc7-popcorn+4
[  389.019389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181124
[  389.019389] Call Trace:
[  389.019389]  dump_stack+0x67/0x90
[  389.019389]  bad_page.cold.36+0x8f/0xc2
[  389.019389]  free_pcppages_bulk+0x18f/0x560
[  389.019389]  free_unref_page_list+0x116/0x240
[  389.019389]  release_pages+0x296/0x2e0
[  389.019389]  tlb_flush_mmu_free+0x31/0x50
[  389.019389]  arch_tlb_finish_mmu+0x2f/0xb0
[  389.019389]  tlb_finish_mmu+0x1f/0x30
[  389.019389]  exit_mmap+0xb1/0x160
[  389.019389]  mmput+0x41/0xf0
[  389.019389]  do_exit+0x38b/0xb50
[  389.019389]  ? _raw_spin_unlock_irq+0x29/0x40
[  389.019389]  do_group_exit+0x39/0xb0
[  389.019389]  __x64_sys_exit_group+0x14/0x20
[  389.019389]  do_syscall_64+0x69/0x400
[  389.019389]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  389.019389]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  389.019389] RIP: 0033:0x7ffff63362e9
[  389.019389] Code: Bad RIP value.
[  389.019389] RSP: 002b:00007fffffffe1d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  389.019389] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007ffff63362e9
[  389.019389] RDX: 00000000000000ff RSI: 0000000000000001 RDI: 00000000000000ff
[  389.019389] RBP: 0000555555813280 R08: 000000000000003c R09: 00000000000000e7
[  389.019389] R10: fffffffffffffe90 R11: 0000000000000246 R12: 00007fffffffe230
[  389.019389] R13: 0000000000000000 R14: 00005555558231d0 R15: 0000555555815e80
[  389.019389] BUG: Bad page state in process sshd  pfn:135ddc
[  389.019389] page:ffffea00043c8820 count:1 mapcount:1 mapping:ffff888139afc001 index:0x8
[  389.019389] flags: 0x8000000000080034(uptodate|lru|active|swapbacked)
[  389.019389] raw: 8000000000080034 dead000000000100 dead000000000200 ffff888139afc001
[  389.019389] raw: 00000007ffff7fe8 0000000000000000 0000000100000000
[  389.019389] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  389.019389] bad because of flags: 0x30(lru|active)
[  389.019389] Modules linked in: msg_socket
[  389.019389] CPU: 0 PID: 589 Comm: sshd Tainted: G    B   W         4.20.0-rc7-popcorn+4
[  389.019389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181124
[  389.019389] Call Trace:
[  389.019389]  dump_stack+0x67/0x90
[  389.019389]  bad_page.cold.36+0x8f/0xc2
[  389.019389]  free_pcppages_bulk+0x18f/0x560
[  389.019389]  free_unref_page_list+0x116/0x240
[  389.019389]  release_pages+0x296/0x2e0
[  389.019389]  tlb_flush_mmu_free+0x31/0x50
[  389.019389]  arch_tlb_finish_mmu+0x2f/0xb0
[  389.019389]  tlb_finish_mmu+0x1f/0x30
[  389.019389]  exit_mmap+0xb1/0x160
[  389.019389]  mmput+0x41/0xf0
[  389.019389]  do_exit+0x38b/0xb50
[  389.019389]  ? _raw_spin_unlock_irq+0x29/0x40
[  389.019389]  do_group_exit+0x39/0xb0
[  389.019389]  __x64_sys_exit_group+0x14/0x20
[  389.019389]  do_syscall_64+0x69/0x400
[  389.019389]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  389.019389]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  389.019389] RIP: 0033:0x7ffff63362e9
[  389.019389] Code: Bad RIP value.
[  389.019389] RSP: 002b:00007fffffffe1d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  389.019389] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007ffff63362e9
[  389.019389] RDX: 00000000000000ff RSI: 0000000000000001 RDI: 00000000000000ff
[  389.019389] RBP: 0000555555813280 R08: 000000000000003c R09: 00000000000000e7
[  389.019389] R10: fffffffffffffe90 R11: 0000000000000246 R12: 00007fffffffe230
[  389.019389] R13: 0000000000000000 R14: 00005555558231d0 R15: 0000555555815e80
[  389.019389] BUG: Bad page state in process sshd  pfn:135ddb
[  389.019389] page:ffffea00043c87e8 count:1 mapcount:1 mapping:ffff888139ae4d49 index:0xe
[  389.019389] flags: 0x8000000000082034(uptodate|lru|active|private|swapbacked)
[  389.019389] raw: 8000000000082034 dead000000000100 dead000000000200 ffff888139ae4d49
[  389.019389] raw: 00000007ffffffbe 0000000000000000 0000000100000000
[  389.019389] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  389.019389] bad because of flags: 0x2030(lru|active|private)
[  389.019389] Modules linked in: msg_socket
[  389.019389] CPU: 0 PID: 589 Comm: sshd Tainted: G    B   W         4.20.0-rc7-popcorn+4
[  389.019389] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-20181124
[  389.019389] Call Trace:
[  389.019389]  dump_stack+0x67/0x90
[  389.019389]  bad_page.cold.36+0x8f/0xc2
[  389.019389]  free_pcppages_bulk+0x18f/0x560
[  389.019389]  free_unref_page_list+0x116/0x240
[  389.019389]  release_pages+0x296/0x2e0
[  389.019389]  tlb_flush_mmu_free+0x31/0x50
[  389.019389]  arch_tlb_finish_mmu+0x2f/0xb0
[  389.019389]  tlb_finish_mmu+0x1f/0x30
[  389.019389]  exit_mmap+0xb1/0x160
[  389.019389]  mmput+0x41/0xf0
[  389.019389]  do_exit+0x38b/0xb50
[  389.019389]  ? _raw_spin_unlock_irq+0x29/0x40
[  389.019389]  do_group_exit+0x39/0xb0
[  389.019389]  __x64_sys_exit_group+0x14/0x20
[  389.019389]  do_syscall_64+0x69/0x400
[  389.019389]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  389.019389]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  389.019389] RIP: 0033:0x7ffff63362e9
[  389.019389] Code: Bad RIP value.
[  389.019389] RSP: 002b:00007fffffffe1d8 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
[  389.019389] RAX: ffffffffffffffda RBX: 00000000000000ff RCX: 00007ffff63362e9
[  389.019389] RDX: 00000000000000ff RSI: 0000000000000001 RDI: 00000000000000ff
[  389.019389] RBP: 0000555555813280 R08: 000000000000003c R09: 00000000000000e7
[  389.019389] R10: fffffffffffffe90 R11: 0000000000000246 R12: 00007fffffffe230
[  389.019389] R13: 0000000000000000 R14: 00005555558231d0 R15: 0000555555815e80
[  389.019657] BUG: Bad page state in process sshd  pfn:135dda
[  389.019835] page:ffffea00043c87b0 count:1 mapcount:1 mapping:ffff88813a342ed9 index:0xf
[  389.020074] flags: 0x8000000000082034(uptodate|lru|active|private|swapbacked)
[  389.020240] raw: 8000000000082034 dead000000000100 dead000000000200 ffff88813a342ed9
[  389.020623] raw: 00000007ffff691f 0000000000000000 0000000100000000
[  389.020927] page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set
[  389.021290] bad because of flags: 0x2030(lru|active|private)
[  389.021579] Modules linked in: msg_socket
bxatnarf commented 5 years ago

The strange thing about stack traces printed in the log is that none of them are for the migrated process -- they are all for sshd or something with PID 51. Nevertheless, I think this is popcorn-related.

AHatnarf commented 4 years ago

Going to close this after 1f12a34a25a122f6b0e512b0326b5b199daf215c. Thanks!