maestro-os / maestro

Unix-like kernel written in Rust
https://blog.lenot.re
GNU Affero General Public License v3.0
2.96k stars 95 forks source link

Integer overflow caused by multiply in `munmap()` #41

Open Marsman1996 opened 2 weeks ago

Marsman1996 commented 2 weeks ago

There is a multiply with overflow problem in kernel/src/syscall/munmap.rs, munmap(), Maestro. The div_ceil() rounds the result towards positive infinity. As a result, when user calls munmap with large length (i.e., 0xfffffff0), the following multiplication operation will cause an integer overflow problem.

https://github.com/maestro-os/maestro/blob/e7ebdfacc22040eeb2dcbe3be17e29c2c3192767/kernel/src/syscall/munmap.rs#L45-L46

For example:

use std::usize;

fn main() {
    let length: usize = usize::MAX;
    let page_size = 0x1000;
    let pages = length.div_ceil(page_size);
    let length = pages * page_size;
    println!("length = 0x{length:x}");
}
thread 'main' panicked at src/main.rs:7:18:
attempt to multiply with overflow
llenotre commented 1 week ago

Thank you for the report! I am putting this on my todolist