search

rr-debugger / rr

Record and Replay Framework
http://rr-project.org/
Other
9.16k stars 587 forks source link

MADV_COLD and MADV_PAGEOUT are unsupported args to madvise. #3702

Closed Nopey closed 8 months ago

Nopey commented 8 months ago

When a program under rr tries to use madvise cold or pageout, rr will (intentionally) crash because it does not support them.

single file reproduction a single-file C program that crashes rr, adapted from the [linux-test-project's madvise01.c test file](https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/syscalls/madvise/madvise01.c) ```c // SPDX-License-Identifier: GPL-2.0-or-later /* * Copyright (c) International Business Machines Corp., 2004 * Copyright (c) Linux Test Project, 2013-2016 */ /* * This is a test case for madvise(2) system call. * It tests madvise(2) with combinations of advice values. * No error should be returned. */ /* * Downloaded from https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/syscalls/madvise/madvise01.c * & then cut down to MADV_COLD and MADV_PAGEOUT, which rr (debugger) doesn't yet support */ #include #include #include #include #include #include #include #include #define TMP_DIR "/tmp" #define TEST_FILE TMP_DIR"/madvtest_testfile" #define KSM_SYS_DIR "/sys/kernel/mm/ksm" #define STR "abcdefghijklmnopqrstuvwxyz12345\n" static char *sfile; static char *amem; static struct stat st; #define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) static struct tcase { int advice; char *name; char **addr; } tcases[] = { #if 0 // rr-supported madvices {MADV_NORMAL, "MADV_NORMAL", &sfile}, {MADV_RANDOM, "MADV_RANDOM", &sfile}, {MADV_SEQUENTIAL, "MADV_SEQUENTIAL", &sfile}, {MADV_WILLNEED, "MADV_WILLNEED", &sfile}, {MADV_DONTNEED, "MADV_DONTNEED", &sfile}, {MADV_REMOVE, "MADV_REMOVE", &sfile}, /* since Linux 2.6.16 */ {MADV_DONTFORK, "MADV_DONTFORK", &sfile}, /* since Linux 2.6.16 */ {MADV_DOFORK, "MADV_DOFORK", &sfile}, /* since Linux 2.6.16 */ {MADV_HWPOISON, "MADV_HWPOISON", &sfile}, /* since Linux 2.6.32 */ {MADV_MERGEABLE, "MADV_MERGEABLE", &sfile}, /* since Linux 2.6.32 */ {MADV_UNMERGEABLE, "MADV_UNMERGEABLE", &sfile}, /* since Linux 2.6.32 */ {MADV_HUGEPAGE, "MADV_HUGEPAGE", &amem}, /* since Linux 2.6.38 */ {MADV_NOHUGEPAGE, "MADV_NOHUGEPAGE", &amem}, /* since Linux 2.6.38 */ {MADV_DONTDUMP, "MADV_DONTDUMP", &sfile}, /* since Linux 3.4 */ {MADV_DODUMP, "MADV_DODUMP", &sfile}, /* since Linux 3.4 */ {MADV_FREE, "MADV_FREE", &amem}, /* since Linux 4.5 */ {MADV_WIPEONFORK, "MADV_WIPEONFORK", &amem}, /* since Linux 4.14 */ {MADV_KEEPONFORK, "MADV_KEEPONFORK", &amem}, /* since Linux 4.14 */ #endif // rr-supported madvices // the following two aren't supported by rr: {MADV_COLD, "MADV_COLD", &amem}, /* since Linux 5.4 */ {MADV_PAGEOUT, "MADV_PAGEOUT", &amem}, /* since Linux 5.4 */ }; static void setup(void) { unsigned int i; int fd; fd = open(TEST_FILE, O_RDWR | O_CREAT, 0664); /* Writing 40 KB of random data into this file [32 * 1280 = 40960] */ for (i = 0; i < 1280; i++) write(fd, STR, strlen(STR)); fstat(fd, &st); /* Map the input file into shared memory */ sfile = mmap(NULL, st.st_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); /* Map the input file into private memory. MADV_HUGEPAGE only works * with private anonymous pages */ amem = mmap(NULL, st.st_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); close(fd); } static void cleanup(void) { munmap(sfile, st.st_size); munmap(amem, st.st_size); unlink(TEST_FILE); } static void verify_madvise(struct tcase *tc) { int result = madvise(*(tc->addr), st.st_size, tc->advice); if (result == -1) { if (result == EINVAL) { printf("%s is not supported (by your kernel)\n", tc->name); } else { printf("madvise test for %s failed with " "return = %d, errno = %d\n", tc->name, result, errno); } } else { printf("madvise test for %s PASSED\n", tc->name); } } int main(void) { setup(); for(unsigned i = 0; i < ARRAY_SIZE(tcases); i++) verify_madvise(&tcases[i]); cleanup(); } ```

As for a real application that I found triggering this, Themaister/Granite with vulkan-swrast-1 24.0.1-1-x86_64 (installed on Arch, this is also known as lavapipe) is the only open source application I've tested that calls madvise with MADV_COLD.

crashlog on Granite's ui-sandbox ``` $ VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/lvp_icd.x86_64.json rr record tests/ui-sandbox rr: Saving execution to trace directory `/home/magnus/.local/share/rr/ui-sandbox-1'. [INFO]: Targeting VK_KHR_present_wait latency to 1 frames. [INFO]: SDL_Init took 0.016 seconds. [INFO]: SDL_Init(GAMEPAD) took 0.081 seconds async. [INFO]: Initializing static assets. [INFO]: Layer count: 6 [INFO]: Found layer: VK_LAYER_RENDERDOC_Capture. [INFO]: Found layer: VK_LAYER_VALVE_steam_fossilize_32. [INFO]: Found layer: VK_LAYER_VALVE_steam_fossilize_64. [INFO]: Found layer: VK_LAYER_VALVE_steam_overlay_32. [INFO]: Found layer: VK_LAYER_VALVE_steam_overlay_64. [INFO]: Found layer: VK_LAYER_KHRONOS_validation. [INFO]: Enabling instance extension: VK_KHR_surface. [INFO]: Enabling instance extension: VK_KHR_wayland_surface. [INFO]: Enabling instance extension: VK_EXT_debug_utils. [INFO]: Enabling instance extension: VK_KHR_get_surface_capabilities2. [INFO]: Found Vulkan GPU: llvmpipe (LLVM 16.0.6, 256 bits) [INFO]: API: 1.3.274 [INFO]: Driver: 0.0.1 [INFO]: Using Vulkan GPU: llvmpipe (LLVM 16.0.6, 256 bits) [INFO]: Enabling device extension: VK_KHR_swapchain. [INFO]: Enabling device extension: VK_EXT_calibrated_timestamps. [INFO]: Enabling device extension: VK_EXT_memory_priority. [INFO]: Enabling device extension: VK_EXT_memory_budget. [INFO]: Enabling device extension: VK_EXT_pageable_device_local_memory. [INFO]: Enabling device extension: VK_NV_device_generated_commands. [INFO]: Enabling device extension: VK_EXT_mesh_shader. [INFO]: Enabling device extension: VK_EXT_index_type_uint8. [INFO]: Enabling device extension: VK_EXT_external_memory_host. [INFO]: Initializing pipeline cache. [FATAL ../src/record_syscall.cc:6312:rec_process_syscall_arch()] (task 14071 (rec:14071) at time 5585) -> Assertion `t->regs().syscall_result_signed() == -syscall_state.expect_errno' failed to hold. Expected EINVAL for 'madvise' but got result 0 (errno SUCCESS); unknown madvise(20) Tail of trace dump: { real_time:58470.062480 global_time:5565, event:`SYSCALLBUF_RESET' tid:14071, ticks:20320632 } { real_time:58470.062499 global_time:5566, event:`SYSCALL: sysinfo' (state:EXITING_SYSCALL) tid:14071, ticks:20320632 rax:0x0 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x7fd4ebb961f0 rsi:0x7ffe29dc0cd0 rdi:0x7ffe29dc0ad0 rbp:0x681fffa0 rsp:0x681ffde0 r8:0x5604a25a5f50 r9:0x5604a25a5f58 r10:0x7 r11:0x246 r12:0x1 r13:0x5604a09e8270 r14:0x7fd4ec415000 r15:0x56049f6511b8 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x63 fs_base:0x7fd4ec2accc0 gs_base:0x0 { tid:14071, addr:0x7ffe29dc0ad0, length:0x70 } } { real_time:58470.062537 global_time:5567, event:`SYSCALLBUF_FLUSH' tid:14071, ticks:20320918 { syscall:'openat', ret:0x12, size:0x10, desched:1 } { syscall:'readlinkat', ret:0xd, size:0x1d } } { real_time:58470.062540 global_time:5568, event:`SYSCALL: fstatat' (state:ENTERING_SYSCALL) tid:14071, ticks:20320918 rax:0xffffffffffffffda rbx:0x7ffe29dc0bc0 rcx:0xffffffffffffffff rdx:0x7ffe29dc0a60 rsi:0x7fd4ebb98bd5 rdi:0x12 rbp:0x7ffe29dc0c50 rsp:0x7ffe29dc0a48 r8:0x0 r9:0x5604a25a5f58 r10:0x1000 r11:0x246 r12:0x3e4aad000 r13:0x12 r14:0x40 r15:0x56049f6511b8 rip:0x7fd4ebafb5da eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x106 fs_base:0x7fd4ec2accc0 gs_base:0x0 } { real_time:58470.062544 global_time:5569, event:`SYSCALLBUF_RESET' tid:14071, ticks:20320918 } { real_time:58470.062576 global_time:5570, event:`SYSCALL: fstatat' (state:EXITING_SYSCALL) tid:14071, ticks:20320918 rax:0x0 rbx:0x7ffe29dc0bc0 rcx:0xffffffffffffffff rdx:0x7ffe29dc0a60 rsi:0x7fd4ebb98bd5 rdi:0x12 rbp:0x7ffe29dc0c50 rsp:0x7ffe29dc0a48 r8:0x0 r9:0x5604a25a5f58 r10:0x1000 r11:0x246 r12:0x3e4aad000 r13:0x12 r14:0x40 r15:0x56049f6511b8 rip:0x7fd4ebafb5da eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x106 fs_base:0x7fd4ec2accc0 gs_base:0x0 { tid:14071, addr:0x7ffe29dc0a60, length:0x90 } } { real_time:58470.062632 global_time:5571, event:`SYSCALLBUF_FLUSH' tid:14071, ticks:20322072 { syscall:'read', ret:0x3f, size:0x4f, desched:1 } { syscall:'read', ret:0x40, size:0x50, desched:1 } { syscall:'read', ret:0x80, size:0x90, desched:1 } { syscall:'read', ret:0x100, size:0x110, desched:1 } { syscall:'read', ret:0x200, size:0x210, desched:1 } { syscall:'read', ret:0x234, size:0x244, desched:1 } { syscall:'read', ret:0x0, size:0x10, desched:1 } { syscall:'close', ret:0x0, size:0x10 } } { real_time:58470.062636 global_time:5572, event:`SYSCALL: sysinfo' (state:ENTERING_SYSCALL) tid:14071, ticks:20322072 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x7fd4ebb961f0 rsi:0x7ffe29dc0cd0 rdi:0x7ffe29dc0ad0 rbp:0x681fffa0 rsp:0x681ffde0 r8:0x1d0 r9:0x0 r10:0x7fd4ebb7fac0 r11:0x246 r12:0x1 r13:0x5604a09e8270 r14:0x7fd4ec415000 r15:0x56049f6511b8 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x63 fs_base:0x7fd4ec2accc0 gs_base:0x0 } { real_time:58470.062638 global_time:5573, event:`SYSCALLBUF_RESET' tid:14071, ticks:20322072 } { real_time:58470.062658 global_time:5574, event:`SYSCALL: sysinfo' (state:EXITING_SYSCALL) tid:14071, ticks:20322072 rax:0x0 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x7fd4ebb961f0 rsi:0x7ffe29dc0cd0 rdi:0x7ffe29dc0ad0 rbp:0x681fffa0 rsp:0x681ffde0 r8:0x1d0 r9:0x0 r10:0x7fd4ebb7fac0 r11:0x246 r12:0x1 r13:0x5604a09e8270 r14:0x7fd4ec415000 r15:0x56049f6511b8 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x63 fs_base:0x7fd4ec2accc0 gs_base:0x0 { tid:14071, addr:0x7ffe29dc0ad0, length:0x70 } } { real_time:58470.062704 global_time:5575, event:`SYSCALLBUF_FLUSH' tid:14071, ticks:20322358 { syscall:'openat', ret:0x12, size:0x10, desched:1 } { syscall:'readlinkat', ret:0xd, size:0x1d } } { real_time:58470.062707 global_time:5576, event:`SYSCALL: fstatat' (state:ENTERING_SYSCALL) tid:14071, ticks:20322358 rax:0xffffffffffffffda rbx:0x7ffe29dc0bc0 rcx:0xffffffffffffffff rdx:0x7ffe29dc0a60 rsi:0x7fd4ebb98bd5 rdi:0x12 rbp:0x7ffe29dc0c50 rsp:0x7ffe29dc0a48 r8:0x0 r9:0x0 r10:0x1000 r11:0x246 r12:0x3e4aad000 r13:0x12 r14:0x40 r15:0x56049f6511b8 rip:0x7fd4ebafb5da eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x106 fs_base:0x7fd4ec2accc0 gs_base:0x0 } { real_time:58470.062709 global_time:5577, event:`SYSCALLBUF_RESET' tid:14071, ticks:20322358 } { real_time:58470.062727 global_time:5578, event:`SYSCALL: fstatat' (state:EXITING_SYSCALL) tid:14071, ticks:20322358 rax:0x0 rbx:0x7ffe29dc0bc0 rcx:0xffffffffffffffff rdx:0x7ffe29dc0a60 rsi:0x7fd4ebb98bd5 rdi:0x12 rbp:0x7ffe29dc0c50 rsp:0x7ffe29dc0a48 r8:0x0 r9:0x0 r10:0x1000 r11:0x246 r12:0x3e4aad000 r13:0x12 r14:0x40 r15:0x56049f6511b8 rip:0x7fd4ebafb5da eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x106 fs_base:0x7fd4ec2accc0 gs_base:0x0 { tid:14071, addr:0x7ffe29dc0a60, length:0x90 } } { real_time:58470.062782 global_time:5579, event:`SYSCALLBUF_FLUSH' tid:14071, ticks:20323658 { syscall:'read', ret:0x3f, size:0x4f, desched:1 } { syscall:'read', ret:0x40, size:0x50, desched:1 } { syscall:'read', ret:0x80, size:0x90, desched:1 } { syscall:'read', ret:0x100, size:0x110, desched:1 } { syscall:'read', ret:0x200, size:0x210, desched:1 } { syscall:'read', ret:0x234, size:0x244, desched:1 } { syscall:'read', ret:0x0, size:0x10, desched:1 } { syscall:'close', ret:0x0, size:0x10 } } { real_time:58470.062786 global_time:5580, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:14071, ticks:20323658 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x4002000 rdi:0x0 rbp:0x681fffa0 rsp:0x681ffde0 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x0 r13:0x0 r14:0x1000 r15:0x7fd4ebc37b20 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fd4ec2accc0 gs_base:0x0 } { real_time:58470.062788 global_time:5581, event:`SYSCALLBUF_RESET' tid:14071, ticks:20323658 } { real_time:58470.062812 global_time:5582, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:14071, ticks:20323658 rax:0x7fd4c7ffe000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x4002000 rdi:0x0 rbp:0x681fffa0 rsp:0x681ffde0 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x0 r13:0x0 r14:0x1000 r15:0x7fd4ebc37b20 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fd4ec2accc0 gs_base:0x0 { map_file:"", addr:0x7fd4c7ffe000, length:0x4002000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x4002000 } } { real_time:58470.062865 global_time:5583, event:`PATCH_SYSCALL' tid:14071, ticks:20323688 rax:0x1c rbx:0x7ffe29dc1060 rcx:0xffffffffffffffff rdx:0x14 rsi:0x4000000 rdi:0x7fd4c7fff000 rbp:0x0 rsp:0x7ffe29dc0ea8 r8:0xffffffff r9:0x0 r10:0x5604a0aef1d0 r11:0x246 r12:0x0 r13:0x7ffe29dc0f60 r14:0x5604a0aef370 r15:0x0 rip:0x7fd4ebb08155 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7fd4ec2accc0 gs_base:0x0 { tid:14071, addr:0x7fd4ebc4769c, length:0x5e } { tid:14071, addr:0x7fd4ebb08155, length:0x8 } } { real_time:58470.062891 global_time:5584, event:`SYSCALL: madvise' (state:ENTERING_SYSCALL) tid:14071, ticks:20323700 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x14 rsi:0x4000000 rdi:0x7fd4c7fff000 rbp:0x681fffa0 rsp:0x681ffde0 r8:0xffffffff r9:0x0 r10:0x5604a0aef1d0 r11:0x246 r12:0x0 r13:0x7ffe29dc0f60 r14:0x5604a0aef370 r15:0x0 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x1c fs_base:0x7fd4ec2accc0 gs_base:0x0 } === Start rr backtrace: rr(_ZN2rr13dump_rr_stackEv+0x3b)[0x5e282b] rr(_ZN2rr9GdbServer15emergency_debugEPNS_4TaskE+0xb6)[0x4bb246] rr(_ZN2rr21EmergencyDebugOstreamD2Ev+0x2a2)[0x4cc5f2] rr[0x5404d9] rr(_ZN2rr19rec_process_syscallEPNS_10RecordTaskE+0x12d)[0x54849d] rr(_ZN2rr13RecordSession21syscall_state_changedEPNS_10RecordTaskEPNS0_9StepStateE+0x8eb)[0x4fcc1b] rr(_ZN2rr13RecordSession11record_stepEv+0x3e5)[0x502745] rr(_ZN2rr13RecordCommand3runERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0xde4)[0x4f6374] rr(main+0x18e)[0x45a21e] /usr/lib/libc.so.6(+0x27c4c)[0x7f251c227c4c] /usr/lib/libc.so.6(__libc_start_main+0x85)[0x7f251c227d05] rr(_start+0x2a)[0x45a36a] === End rr backtrace Launch gdb with gdb '-l' '10000' '-ex' 'set sysroot /' '-ex' 'target extended-remote 127.0.0.1:14071' /home/magnus/Granite/build/tests/ui-sandbox ```
rocallahan commented 8 months ago

Have you tried with rr master? MADV_COLD should work there.

Nopey commented 8 months ago

That's fantastic news! I haven't, will do

Feb 29, 2024 14:42:22 rocallahan @.***>:

Have you tried with rr master? MADV_COLD should work there.

— Reply to this email directly, view it on GitHub[https://github.com/rr-debugger/rr/issues/3702#issuecomment-1972095841], or unsubscribe[https://github.com/notifications/unsubscribe-auth/ABCVJ6ZQKKV6BPTMDO44TSLYV6XE5AVCNFSM6AAAAABEAW7JB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZSGA4TKOBUGE]. You are receiving this because you authored the thread. [Tracking image][https://github.com/notifications/beacon/ABCVJ635FUASCLGKLXLROOTYV6XE5A5CNFSM6AAAAABEAW7JB2WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTVRPFWC.gif]

Nopey commented 8 months ago

Thanks again, it works!