Node Error on deflateData with big files

[siem4]

I have an ifc file of 385 mB ±. If I try to convert it with the following command:

node --max-old-space-size=20000 convert2xkt.js -s"file.ifc" -o"geometry.xkt" -l

I get the following error:

[convert2xkt] Running convert2xkt v1.1.22...
[convert2xkt] Using configs in ./convert2xkt.conf.js
[convert2xkt] Converting single input file /var/rails_apps/fouriq/poc's research/xeokit-bim-viewer/app/data/projects/RAC/models/rac/O-S1-BWK-BIM architectural - BIM bouwkundig.ifc...
[convert2xkt] Reading input file: /var/rails_apps/fouriq/poc's research/xeokit-bim-viewer/app/data/projects/RAC/models/rac/O-S1-BWK-BIM architectural - BIM bouwkundig.ifc
[convert2xkt] Input file extension: "ifc"
[convert2xkt] [WARNING] Could not find configs sourceConfigs entry for source format "ifc". This is derived from the source file name extension. Will use internal default configs.
[convert2xkt] Input file size: 394697.40 kB
[convert2xkt] Not embedding metadata in XKT
[convert2xkt] Using parser: parseIFCIntoXKTModel
INFO:  Parsing Model using IFC2X3 Schema
[convert2xkt] Creating default metamodel in XKT
[convert2xkt] Input file parsed OK. Building XKT document...
[convert2xkt] XKT document built OK. Writing to XKT file...
hi1
hi2
hi3
1

#
# Fatal error in , line 0
# Fatal JavaScript invalid size error 169220804 (see crbug.com/1201626)
#
#
#
#FailureMessage Object: 0x7ffedc093680
----- Native stack trace -----

 1: 0x55fef4177bf5  [node]
 2: 0x55fef50f6176 V8_Fatal(char const*, ...) [node]
 3: 0x55fef44ec32a  [node]
 4: 0x55fef46a72ef  [node]
 5: 0x55fef46a82ee  [node]
 6: 0x55fef48d107c v8::internal::Runtime_GrowArrayElements(int, unsigned long*, v8::internal::Isolate*) [node]
 7: 0x55fe94c9a2b6 
Trace/breakpoint trap (core dumped)

The hi comments are logs to test where the error occurs😨, please don't get mad… If there is not enough ram available, it will just log killed instead of the stack trace. The 1 is a counter in the deflate function in deflateData.

The system that was used is Rocky Linux with 20Gb ram, the 20Gb limit is never reached.

The error comes from deflateData in writeXKTModelToArrayBuffer. Possible causes:

• I suspect it is writing too much data to the buffer that makes Nodejs crash.
• Or that there are to many recursion calls in deflateData that make it hit a limit, I have tried with node --stack-size to 1000000, and that didn't fix it. And there is no recursion in the deflate function.

Please let me know if there is any extra info that is required.

[xeolabs]

Hi, this looks like the 3rd-party web-ifc library hitting the limits of how much geometry it can convert.

We plan to deprecate the direct IFC->XKT conversion mode, which uses the web-ifc library.

Which version of xeokit-convert are you using? Possibly you're using an old version - please try the latest version and see if it works for this model.

The process we recommend for IFC->XKT conversion is described here: https://xeokit.github.io/xeokit-model-conversion-tests/index.html

That's a much more performant method, which requires an additional ifc2gltf CLI tool - please find all the details at that link.

[siem4]

I'm currently running version v1.1.22

I will look into the different conversion technique, my understanding is that using the Creeox AG converter is not permitted to use without a proper license for company's. Therefore, I was hoping to be using the ifc -> xkt converter

[xeolabs]

There's also this option for conversion, which does not use ifc2gltf: https://www.notion.so/xeokit/Converting-IFC-Models-to-XKT-using-Open-Source-Tools-A-Simpler-Pipeline-02d45ba457eb4f808f63bcacb71a4fb3

We recommend the IFC->ifc2gltf->glTF->convert2xkt->XKT pipeline for enterprises, it's much better.

[siem4]

Thank you for the response, I understand that it might be better for enterprise, what is the big reason that for enterprise use it might be better to use the licensed version?

[xeolabs]

ifc2gltf performs much better with large IFC files because it's able to split them into a set of glTF files, and convert2xkt is able to convert those into a set of split XKT files. This means that converters and viewer are not overwhelmed by memory requirements of processing large IFC files.

Also, ifc2gltf converts the IFC to higher-quality geometry. We maintain that tool, but have no control over the web-ifc library.