Custom oom handler that calls panic instead of abort

robert-w-gries commented 6 years ago

Currently, the Heap and LockedHeap uses the default oom handler, which calls intrinsics::abort(). On the x86_64 architecture, this generates a ud2 instruction, generating an invalid opcode exception.

While using this library in a kernel, allocating too much memory will call the default oom handler and generate an INVALID OPCODE exception that points to the first ud2 instruction in alloc::oom (0x1073fc in my case):

0000000000107ef0 <_ZN5alloc9allocator5Alloc3oom17h5474e96b9ba411a5E>:
  107ef0:   55                      push   %rbp
  107ef1:   48 89 e5                mov    %rsp,%rbp
  107ef4:   48 83 ec 10             sub    $0x10,%rsp
  107ef8:   48 89 7d f8             mov    %rdi,-0x8(%rbp)
 >107efc:   0f 0b                   ud2    
  107efe:   0f 0b                   ud2    
  107f00:   48 89 75 f0             mov    %rsi,-0x10(%rbp)
  107f04:   66 2e 0f 1f 84 00 00    nopw   %cs:0x0(%rax,%rax,1)
  107f0b:   00 00 00 
  107f0e:   66 90                   xchg   %ax,%ax

I find the exception message unhelpful. So instead of generating an exception, we can set a custom oom handler by implementing oom for Heap and LockedHeap. Custom oom handlers will be important later when we want to gracefully handle errors. My PR results in the following message when we run out of memory.

PANIC in /path/to/linked-list-allocator/src/lib.rs at line 180:
    Out of memory

Testing

You can test this PR by adding the following to src/lib.rs of blog_os

let mut stack: alloc::Vec<usize> = vec![0; 100_000_000];

phil-opp commented 6 years ago

Thanks a lot!

phil-opp commented 6 years ago

Published as 0.4.3

rust-osdev / linked-list-allocator

Custom oom handler that calls panic instead of abort #7

Testing