search

ziglang / zig

General-purpose programming language and toolchain for maintaining robust, optimal, and reusable software.
https://ziglang.org
MIT License
34.7k stars 2.53k forks source link

reader.readAllAlloc crash on Ampere a1 CPU when given a much larger max_size parameter than the actual file size #20714

Open WeijieH opened 3 months ago

WeijieH commented 3 months ago

Zig Version

0.14.0-dev.367+a57479afc

Steps to Reproduce and Observed Behavior

Example zig code:

const std = @import("std");
const fs = std.fs;
const print = std.debug.print;

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();
    const allocator = gpa.allocator();

    const file = try fs.cwd().openFile("crash.txt", .{});
    defer file.close();

    var buf_reader = std.io.bufferedReader(file.reader());
    const reader = buf_reader.reader();

    const buffer = try reader.readAllAlloc(allocator, 10240);
    defer allocator.free(buffer);

    print("{s}\n", .{buffer});
}

And crash.txt with only 5 letters:

crash

And run $ zig run example.zig will result in a crash:

thread 2442597 panic: reached unreachable code
/home/opc/zig-linux-aarch64-0.14.0-dev.367+a57479afc/lib/std/posix.zig:4699:19: 0x104fa9f in munmap (t)
        .INVAL => unreachable, // Invalid parameters.
                  ^
/home/opc/zig-linux-aarch64-0.14.0-dev.367+a57479afc/lib/std/heap/PageAllocator.zig:94:21: 0x104f9c3 in resize (t)
        posix.munmap(@alignCast(ptr[0 .. buf_aligned_len - new_size_aligned]));
                    ^
/home/opc/zig-linux-aarch64-0.14.0-dev.367+a57479afc/lib/std/mem/Allocator.zig:92:30: 0x107a923 in resizeLarge (t)
    return self.vtable.resize(self.ptr, buf, log2_buf_align, new_len, ret_addr);
                             ^
/home/opc/zig-linux-aarch64-0.14.0-dev.367+a57479afc/lib/std/heap/general_purpose_allocator.zig:722:40: 0x104fecf in resize (t)
                return self.resizeLarge(old_mem, log2_old_align, new_size, ret_addr);
                                       ^
/home/opc/zig-linux-aarch64-0.14.0-dev.367+a57479afc/lib/std/mem/Allocator.zig:92:30: 0x107d6e3 in resize__anon_7795 (t)
    return self.vtable.resize(self.ptr, buf, log2_buf_align, new_len, ret_addr);
                             ^
/home/opc/zig-linux-aarch64-0.14.0-dev.367+a57479afc/lib/std/array_list.zig:1010:33: 0x1096e6f in shrinkAndFree (t)
            if (allocator.resize(old_memory, new_len)) {
                                ^
/home/opc/zig-linux-aarch64-0.14.0-dev.367+a57479afc/lib/std/array_list.zig:398:36: 0x108b5bb in shrinkAndFree (t)
            unmanaged.shrinkAndFree(self.allocator, new_len);
                                   ^
/home/opc/zig-linux-aarch64-0.14.0-dev.367+a57479afc/lib/std/io/Reader.zig:76:37: 0x107d443 in readAllArrayListAligned__anon_7794 (t)
            array_list.shrinkAndFree(start_index);
                                    ^
/home/opc/zig-linux-aarch64-0.14.0-dev.367+a57479afc/lib/std/io/Reader.zig:52:40: 0x1052363 in readAllArrayList (t)
    return self.readAllArrayListAligned(null, array_list, max_append_size);
                                       ^
/home/opc/zig-linux-aarch64-0.14.0-dev.367+a57479afc/lib/std/io/Reader.zig:92:30: 0x104c60f in readAllAlloc (t)
    try self.readAllArrayList(&array_list, max_size);
                             ^
/home/opc/zig-linux-aarch64-0.14.0-dev.367+a57479afc/lib/std/io.zig:135:54: 0x104bf7f in main (t)
            return @errorCast(self.any().readAllAlloc(allocator, max_size));
                                                     ^
/home/opc/zig-linux-aarch64-0.14.0-dev.367+a57479afc/lib/std/start.zig:532:37: 0x104bd1f in posixCallMainAndExit (t)
            const result = root.main() catch |err| {
                                    ^
???:?:?: 0x0 in ??? (???)
Aborted (core dumped)

To avoid crash, I can either change const buffer = try reader.readAllAlloc(allocator, 10240); to const buffer = try reader.readAllAlloc(allocator, 1024); or read a different file with more text in it. For example, the same text file but with with 999 lines of 'crash'

I only observed this behavior on my Oracle Cloud Ampere A1 CPU machines. I don't have other aarch64 machines to test so I'm not sure if this is Ampere A1 CPU only. Same code will run on x64 windows and x64 linux without any problems.

Expected Behavior

readAllAlloc should read the file without crash

gcoakes commented 2 months ago

Could not reproduce with -target aarch64-linux via qemu-user:

details 
$ zig version
0.14.0-dev.367+a57479afc
$ zig run -target aarch64-linux repro.zig
crash

The following lines from the original report seem to indicate a syscall gave the OS an unexpected value.

/home/opc/zig-linux-aarch64-0.14.0-dev.367+a57479afc/lib/std/posix.zig:4699:19: 0x104fa9f in munmap (t)
        .INVAL => unreachable, // Invalid parameters.

Linux's manpage for munmap indicates pointer must be aligned to the page size whereas length does not.

@WeijieH, could you please do the following:

  1. Clarify what OS you're using.
  2. Run and report the output of (EDIT: added PAGE_SIZE print):
$ zig build-exe repro.zig
$ strace --trace=munmap,mmap ./repro
$ getconf PAGE_SIZE
alexrp commented 2 months ago

Very strongly suspect this is a duplicate of #16331, or closely related.

WeijieH commented 2 months ago

Could not reproduce with -target aarch64-linux via qemu-user:

details The following lines from the original report seem to indicate a syscall gave the OS an unexpected value.

/home/opc/zig-linux-aarch64-0.14.0-dev.367+a57479afc/lib/std/posix.zig:4699:19: 0x104fa9f in munmap (t)
        .INVAL => unreachable, // Invalid parameters.

Linux's manpage for munmap indicates pointer must be aligned to the page size whereas length does not.

@WeijieH, could you please do the following:

  1. Clarify what OS you're using.
  2. Run and report the output of (EDIT: added PAGE_SIZE print):
$ zig build-exe repro.zig
$ strace --trace=munmap,mmap ./repro
$ getconf PAGE_SIZE

Here is the result of strace:

[opc@instance-20221208-2328 test]$ strace --trace=munmap,mmap ./t
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xfffeef910000
mmap(0xfffeef911000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xfffeef900000
munmap(0xfffeef901000, 4096)            = -1 EINVAL (Invalid argument)
thread 3648566 panic: reached unreachable code
mmap(0xfffeef902000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xfffeef8f0000
mmap(NULL, 2554264, PROT_READ, MAP_SHARED, 4, 0) = 0xfffeef680000
mmap(0xfffeef8f1000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xfffeef670000
mmap(0xfffeef672000, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xfffeef660000
mmap(0xfffeef664000, 45056, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xfffeef650000
mmap(0xfffeef65b000, 147456, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xfffeef620000
mmap(0xfffeef644000, 495616, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xfffeef5a0000
/home/opc/zig/0.14.0-dev.764+eb1a199df/files/lib/std/posix.zig:4731:19: 0x104e6f7 in munmap (t)
        .INVAL => unreachable, // Invalid parameters.
                  ^
/home/opc/zig/0.14.0-dev.764+eb1a199df/files/lib/std/heap/PageAllocator.zig:94:21: 0x104e61b in resize (t)
        posix.munmap(@alignCast(ptr[0 .. buf_aligned_len - new_size_aligned]));
                    ^
/home/opc/zig/0.14.0-dev.764+eb1a199df/files/lib/std/mem/Allocator.zig:92:30: 0x1078f4f in resizeLarge (t)
    return self.vtable.resize(self.ptr, buf, log2_buf_align, new_len, ret_addr);
                             ^
/home/opc/zig/0.14.0-dev.764+eb1a199df/files/lib/std/heap/general_purpose_allocator.zig:722:40: 0x104eb27 in resize (t)
                return self.resizeLarge(old_mem, log2_old_align, new_size, ret_addr);
                                       ^
/home/opc/zig/0.14.0-dev.764+eb1a199df/files/lib/std/mem/Allocator.zig:92:30: 0x107bbcf in resize__anon_7814 (t)
    return self.vtable.resize(self.ptr, buf, log2_buf_align, new_len, ret_addr);
                             ^
/home/opc/zig/0.14.0-dev.764+eb1a199df/files/lib/std/array_list.zig:1010:33: 0x109546b in shrinkAndFree (t)
            if (allocator.resize(old_memory, new_len)) {
                                ^
/home/opc/zig/0.14.0-dev.764+eb1a199df/files/lib/std/array_list.zig:398:36: 0x1089cc7 in shrinkAndFree (t)
            unmanaged.shrinkAndFree(self.allocator, new_len);
                                   ^
/home/opc/zig/0.14.0-dev.764+eb1a199df/files/lib/std/io/Reader.zig:76:37: 0x107b92f in readAllArrayListAligned__anon_7813 (t)
            array_list.shrinkAndFree(start_index);
                                    ^
/home/opc/zig/0.14.0-dev.764+eb1a199df/files/lib/std/io/Reader.zig:52:40: 0x1050fbb in readAllArrayList (t)
    return self.readAllArrayListAligned(null, array_list, max_append_size);
                                       ^
/home/opc/zig/0.14.0-dev.764+eb1a199df/files/lib/std/io/Reader.zig:92:30: 0x104c067 in readAllAlloc (t)
    try self.readAllArrayList(&array_list, max_size);
                             ^
/home/opc/zig/0.14.0-dev.764+eb1a199df/files/lib/std/io.zig:135:54: 0x104b9d7 in main (t)
            return @errorCast(self.any().readAllAlloc(allocator, max_size));
                                                     ^
/home/opc/zig/0.14.0-dev.764+eb1a199df/files/lib/std/start.zig:570:37: 0x104b777 in posixCallMainAndExit (t)
            const result = root.main() catch |err| {
                                    ^
???:?:?: 0x0 in ??? (???)
--- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=3648566, si_uid=1000} ---
+++ killed by SIGABRT (core dumped) +++
Aborted (core dumped)

PAGE_SIZE is 65536

system OS is oracle linux 8 Linux instance-20221208-2328 5.4.17-2136.333.5.el8uek.aarch64 #3 SMP Thu Jun 20 01:09:36 PDT 2024 aarch64 aarch64 aarch64 GNU/Linux

gcoakes commented 2 months ago

@WeijieH , I think that confirms what @alexrp said. Misaligned munmap due to an unexpected page size for your platform.