Out of memory error isn't reported in the shell in certain conditions

[Kristopher38]

Version: Minecraft - 1.12, OC - 1.7.5.201, 64-bit natives

Description: When Lua tries to allocate memory for a very large table that won't fit into the computer's RAM anymore, it throws "out of memory" error but it's not displayed in the shell if it isn't catched with pcall/xpcall. The error can still be catched manually with pcall, but the lack of any information printed sometimes can leave the user wondering why the script suddenly stopped working with no error traceback.

Repro: Run the following script on a computer with two tier 3.5 RAM sticks and OpenOS running:

local t = {}
local limit = 2^17 + 1
for i = 1, limit do
    t[i] = i
end
print("ok")

The way it works is, Lua allocates space for table elements in amounts being powers of two. By adding 1 to 2^17, after filling up 2^17 elements in a table, we force Lua to allocate a chunk of memory for the next 2^17 elements, which causes "out of memory" error (if we didn't add 1 at the end, the script would print "ok". If it actually does print "ok" when trying to repro this, try using a larger power of 2, but 2^17 should work). The error isn't printed in the shell though and it makes it seem like the script exited without errors (where in fact it should print "ok" if it actually did). The error can still be catched with pcall like so:

local t = {}
local limit = 2^17 + 1
for i = 1, limit do
    local status, err = pcall(function() t[i] = i end)
    if not status then print(err) end
end
print("ok")

Here it prints both "not enought memory" and "ok", and exits gracefully.

When not catched by pcall/xpcall though, it should be fixed to display the error traceback in the shell.

[payonel]

this happens because in an out-of-memory situation, the process library can't format the traceback and in fact, a log of the crash is written to /tmp/event.log - but maybe in this extreme case it makes more sense to print a message about the crash to stderr

[payonel]

doing some testing, i'll have the process exception handler print to stderr this is a general handler in openos, so this will be the output of this type of crash process library exception handler crashed: %s where %s in this case is "not enough memory"