Long-life arrays leak memory in various places

[dktapps]

Issue description

PHP "arrays" do not release memory when unset()ing elements from them, ever. This is problematic for long-life arrays in a long-life program like PocketMine-MP because memory allocated in, for example, the level chunks array will never be released.

The below code can be used to verify the problem.

$a = [];
for($i = 0; $i < 1000000; ++$i){
    $a[$i] = $i;
}
$fullArray = memory_get_usage();
var_dump(number_format($fullArray));
for($i = 0; $i < 1000000; ++$i){
    unset($a[$i]);
}
$afterUnset = memory_get_usage();
var_dump("memory freed (unset): " . number_format($fullArray - $afterUnset));
$a = [];
$afterDealloc = memory_get_usage();
var_dump("memory freed (destroy): " . number_format($afterUnset - $afterDealloc));

This appears to be an artifact of the internal array implementation. It makes sense in a web server environment, but not in a CLI program.

Possible solutions

• Reallocate known arrays on GC to reclaim lost memory. This would require a lot of boilerplate code which would not be self-explanatory to outside contributors.
• Replace arrays with alternative structures like ds datastructures. As documented here, a Ds\Map object releases memory when the allocated slots drop below 1/4 of the capacity. This could have adverse side effects on performance in a loop like this one due to repeated reallocations, so this needs to be carefully handled. (In the script above, replacing the array with a Map frees the memory appropriately, but is 60% slower).

[dktapps]

Sadly ext-ds is not a solution to this problem because its behaviour when modifying a datastructure during iteration is undefined, leading to bugs.

[dktapps]

To memory, the only major place where this bug might have a significant is in World->chunks, and a specific workaround for that might be OK. Almost all other long-life arrays in PM code don't have large variations in size.