Fix Unicorn Engine 1GB limit that calls exit: raise OSError instead (Fixes #2343)

This commit fixes #2343: an issue where pwn checksec <binary> would fail with a bogus error of:

$ pwn checksec /root/x/bin/main
Could not allocate dynamic translator buffer

This actually comes from Unicorn Engine which tries to allocate a 1GB RWX mapping:

root@pwndbg:~# strace -e openat,mmap pwn checksec /root/x/bin/main 2>&1 | tail -n 10
openat(AT_FDCWD, "/usr/lib/python3/dist-packages/mpmath-0.0.0.egg-info/PKG-INFO", O_RDONLY|O_CLOEXEC) = 7
openat(AT_FDCWD, "/usr/local/lib/python3.10/dist-packages/unicorn/lib/libunicorn.so.2", O_RDONLY|O_CLOEXEC) = 7
mmap(NULL, 22447520, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 7, 0) = 0x7f2604f9d000
mmap(0x7f2605339000, 13496320, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x39c000) = 0x7f2605339000
mmap(0x7f2606018000, 3039232, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x107b000) = 0x7f2606018000
mmap(0x7f26062fe000, 1601536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 7, 0x1360000) = 0x7f26062fe000
mmap(0x7f2606485000, 525728, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2606485000
mmap(NULL, 1073741824, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
Could not allocate dynamic translator buffer
+++ exited with 1 +++

...and if it fails, it calls exit(). This can be seen in Unicorn Engine code: https://github.com/unicorn-engine/unicorn/blob/56f3bdedb42d26bee1532cc01baf5eaf44a9aa23/qemu/accel/tcg/translate-all.c#L960-L963

This issue has been reported to Unicorn Engine in https://github.com/unicorn-engine/unicorn/issues/1766 but since it hasn't been fixed, this commit applies a workaround for it.

Gallopsled / pwntools

Fix Unicorn Engine 1GB limit that calls exit: raise OSError instead (Fixes #2343) #2347