mmtk / mmtk-core

Memory Management ToolKit
https://www.mmtk.io
Other
379 stars 69 forks source link

Panic with 'add with overflow' in fastpath instead of OOM #915

Open qinsoon opened 1 year ago

qinsoon commented 1 year ago

There is an issue with this PR: if we try to do any address arithmetic in the allocation fastpath, we may have overflow in the computation, and we do not call out_of_memory() for that case. See the test below. I don't think we have a good way to deal with this, given that we do not want to introduce any check in the fastpath. https://github.com/mmtk/mmtk-core/blob/04c0c48b3e0c6710bd2308c3477006cd070b4ef3/vmbindings/dummyvm/src/tests/issue867_allocate_unrealistically_large_object.rs#L27

Originally posted by @qinsoon in https://github.com/mmtk/mmtk-core/issues/896#issuecomment-1678477353

qinsoon commented 1 year ago

@k-sareen suggested

One alternative, although I don't know how feasible it is, would be to register a custom panic handler and then call either out_of_memory or panic_in_vm (random function) or something to allow the VM binding to crash by itself. We discussed doing something similar in the OOM PR and https://github.com/mmtk/mmtk-core/issues/746