Fatal JavaScript invalid size error 169220804

[intrida-dev]

Describe the bug We are trying to create XKT file from our own JSON which contains graphical + metadata information. We are able to create xktModel and also finalize() it. But when we try to get an array buffer for this xktModel using writeXKTModelToArrayBuffer, it fails. It shows following error.

Fatal JavaScript invalid size error 169220804

Below is the code we are using to create XKT from JSON data. async function createModel() { let inputJson = readJson(process.argv[2]);

xktModel = new XKTModel();

// root metaobject
xktModel.createMetaObject({ // Root XKTMetaObject, has no XKTEntity
    metaObjectId: "root",
    metaObjectName: "root",
    metaObjectType: "root"
});

if ("entities" in inputJson) {
    let entities = inputJson["entities"];

    for (let i = 0; i < entities.length; i++) {
        createItem(entities[i], "root");
    }
}

await xktModel.finalize();
const xktArrayBuffer = writeXKTModelToArrayBuffer(xktModel);

// write xkt file
fs.writeFile(process.argv[3], Buffer.from(xktArrayBuffer), function (err) {
    if (err) {
        console.log("Error in creating XKT");
    } else {
        console.log("XKT created successfully");
    }
});

}

Note: we are using xeokit-convert ^1.1.3.

Are we doing something wrong here? Please let us know how to handle this problem.

Screenshots

[xeolabs]

From what I can tell, I see nothing wrong in your code. I wonder if it's a memory limit issue - does it work with a smaller number of objects? Would it be possible to get a simple node script that reproduces this issue?

[intrida-dev]

From what I can tell, I see nothing wrong in your code. I wonder if it's a memory limit issue - does it work with a smaller number of objects? Would it be possible to get a simple node script that reproduces this issue?

Yes, it does work for smaller number of objects. I have shared the script files and input json file to reproduce the same through email.

[intrida-dev]

I can't send you test files via mail. It got rejected. I am sending you test data here. test data.zip

Please use following command to run it.

node ".\XktCreator.js" ".\input.json" ".\output.xkt"

[xeolabs]

The error means you're trying to create a JS array with too many elements. Arrays are capped at about 140 million elements.

In other words, you're hitting the memory limits of NodeJS.

Instead of inserting the metaobjects into the XKT, I'd recommend building a JSON file containing the metaobjects instead, so as to have the metadata in that JSON file, alongside the XKT.

Note that we support the option to have the geometry and materials in the XKT, and the metadata in a separate JSON file.

We actually recommend doing that, rather than stuffing all the geometry, materials and metadata into the XKT file. That seemed like a good idea once, so we supported it, just to simplify model pipelines and reduce the number of HTTP GETs when loading models, but now it seems best to keep separate XKT and JSON files.

[intrida-dev]

The error means you're trying to create a JS array with too many elements. Arrays are capped at about 140 million elements.

In other words, you're hitting the memory limits of NodeJS.

Instead of inserting the metaobjects into the XKT, I'd recommend building a JSON file containing the metaobjects instead, so as to have the metadata in that JSON file, alongside the XKT.

Note that we support the option to have the geometry and materials in the XKT, and the metadata in a separate JSON file.

We actually recommend doing that, rather than stuffing all the geometry, materials and metadata into the XKT file. That seemed like a good idea once, so we supported it, just to simplify model pipelines and reduce the number of HTTP GETs when loading models, but now it seems best to keep separate XKT and JSON files.

Thanks for your reply. We can consider having metadata in separate JSON file. We will get back to you if we come across any other issue.