Fatal memory overflow in specific seeds in C++

[Desperationis]

I don't know how to explain this. I've literally debugged this for days, even rewrote my multi-file project into a one-file one, and I am still not sure what causes it. I am on Linux, and program in the C++ v1.1.x kit.

Basically, there are certain seeds that cause my bot to crash in one gigantic memory overflow, which is always near turn 340-360. The error always looks like this, though memory access out of bounds can also happen:

[Agent 0 Log] Cannot enlarge memory arrays to size 16781312 bytes (OOM). Either (1) compile with  -s INITIAL_MEMORY=X  with X higher than the current value 16777216, (2) compile with  -s ALLOW_MEMORY_GROWTH=1  which allows increasing the size at runtime, or (3) if you want malloc to return NULL (0) instead of this abort, compile with  -s ABORTING_MALLOC=0 
[Agent 0 Log] exception thrown: RuntimeError: abort(Cannot enlarge memory arrays to size 16781312 bytes (OOM). Either (1) compile with  -s INITIAL_MEMORY=X  with X higher than the current value 16777216, (2) compile with  -s ALLOW_MEMORY_GROWTH=1  which allows increasing the size at runtime, or (3) if you want malloc to return NULL (0) instead of this abort, compile with  -s ABORTING_MALLOC=0 ) at Error
[Agent 0 Log]     at jsStackTrace (/home/diego/Desktop/LuxBot/main.js:1807:19)
[Agent 0 Log]     at stackTrace (/home/diego/Desktop/LuxBot/main.js:1824:16)
[Agent 0 Log]     at abort (/home/diego/Desktop/LuxBot/main.js:1543:44)
[Agent 0 Log]     at abortOnCannotGrowMemory (/home/diego/Desktop/LuxBot/main.js:1931:7)
[Agent 0 Log]     at _emscripten_resize_heap (/home/diego/Desktop/LuxBot/main.js:1936:7)
[Agent 0 Log]     at <anonymous>:wasm-function[3226]:0x60e6e
[Agent 0 Log]     at malloc (<anonymous>:wasm-function[3220]:0x5edc8)
[Agent 0 Log]     at <anonymous>:wasm-function[3002]:0x5acdc
[Agent 0 Log]     at <anonymous>:wasm-function[102]:0x6b50
[Agent 0 Log]     at <anonymous>:wasm-function[57]:0x41ba,RuntimeError: abort(Cannot enlarge memory arrays to size 16781312 bytes (OOM). Either (1) compile with  -s INITIAL_MEMORY=X  with X higher than the current value 16777216, (2) compile with  -s ALLOW_MEMORY_GROWTH=1  which allows increasing the size at runtime, or (3) if you want malloc to return NULL (0) instead of this abort, compile with  -s ABORTING_MALLOC=0 ) at Error

Keep in mind that this only happens rarely, like 1 in 50 seeds.

Here is a one file version of my bot using the cpp kit in the v1.1.x branch:

#include "lux/kit.hpp"
#include "lux/define.cpp"
#include <string.h>
#include <vector>
#include <set>
#include <stdio.h>
#include <map>

using namespace std;
using namespace lux;
int main()
{
    kit::Agent gameState;
    gameState.initialize();

    while (true)
    {
        GameMap& gameMap = gameState.map;
        Player& player = gameState.players[gameState.id];
        vector<string> actions;
        gameState.update();

        vector<Cell*> resourceTiles;
        for (int y = 0; y < gameMap.height; y++)
        {
            for (int x = 0; x < gameMap.width; x++)
            {
                Cell *cell = gameMap.getCell(x, y);
                if (cell->hasResource())
                {
                    resourceTiles.push_back(cell);
                }
            }
        }

        for (int i = 0; i < player.units.size(); i++)
        {
            Unit& unit = player.units[i];
            if(unit.isWorker() && unit.canAct()) {

                if(unit.getCargoSpaceLeft() != 0) {
                    ResourceType resourceToFind = ResourceType::wood;

                    if(player.researchedCoal()) resourceToFind = ResourceType::coal;
                    if(player.researchedUranium()) resourceToFind = ResourceType::uranium;

                    Cell* closestResourceTile = nullptr;
                    float closestDist = 999999;

                    for(auto it = resourceTiles.begin(); it != resourceTiles.end(); it++) {
                        Cell* cell = *it;
                        float dist = cell->pos.distanceTo(unit.pos);
                        if(cell->resource.type == resourceToFind && dist < closestDist) {
                            closestDist = dist;
                            closestResourceTile = cell;
                        }
                    }

                    DIRECTIONS dir = unit.pos.directionTo(closestResourceTile->pos);
                    actions.push_back(unit.move(dir));
                }
                else {
                    CityTile* closestCityTile = nullptr;
                    float closestDist = 999999;

                    for(auto city : player.cities) {
                        for(auto cityTile : city.second->citytiles) {
                            float dist = cityTile->pos.distanceTo(unit.pos);

                            if(dist < closestDist) {
                                closestCityTile = cityTile; closestDist = dist;
                            }
                        }
                    }
                    DIRECTIONS dir = unit.pos.directionTo(closestCityTile->pos);
                    actions.push_back(unit.move(dir));
                }
            }
        }

        for (int i = 0; i < actions.size(); i++)
        {
            if (i != 0)
                cout << ",";
            cout << actions[i];
        }
        cout << endl;
        // end turn
        gameState.end_turn();
    }

    return 0;
}

To get the same error, you must crosscompile main.cpp into a JS file then run the main.js it against itself in seed 556459304 using the v1.1.x cpp kit.

Here are noteworthy findings:

• You don't get this error if you run the cpp file directly. If you manage to compile cpp as discussed in #71, you never get this type of error.
• The error seems to indicate the program only has like 16mb of memory.
• Replay doesn't show anything noteworthy. I spent hours analyzing the last movements of the program before it crashed and nothing seemed wrong.
• Happens rarely, like 1 in 50 odds
• The program always, I mean ALWAYS, crashes when you call kit::Agent.update(). Debugging via cerr showed that the entire program run fine with no issues whatsoever right up until kit::Agent.update().

[StoneT2000]

Looks like adding -s INITIAL_MEMORY=X to the compile command is sufficient. Had the error you mentioned earlier but it was gone after adding this.

We will make this a default in the standard c++ kit. Let me know if setting X to something like 128mb works, so do

-s INITIAL_MEMORY=134217728

Full compile command is

emcc -s FORCE_FILESYSTEM=1 -s INITIAL_MEMORY=134217728 --pre-js internals/init_fs.js main.cpp -o main.js

[Desperationis]

@StoneT2000 That did the trick! Thanks :)

[StoneT2000]

fixed in #51