Creoox / creoox-ifc2gltfcxconverter

Public repository of Creoox & XeoKit convertion Tool
Other
14 stars 2 forks source link

Large number of empty .glb files created as output for some IFC models #7

Closed xeolabs closed 5 months ago

xeolabs commented 6 months ago

ifc2gltfcxconverter 3_0_19_beta on Linux creates thousands of output .glb files for some IFC models

Example:

ifc2gltfcxconverter -i AR-Demo_Sample_Single_Building_01.ifc -o model.glb -m model.json -s 5 -t 200 -e 3

In this case it outputs 9999 .glb files.

The contents of each file looks like this:

glTF   P  <  JSON{"asset":{"generator":"ConverterGLTF","version":"2.0"},"bufferViews":[{"buffer":0,"byteLength":0,"byteStride":12,"target":34962},{"buffer":0,"byteLength":0,"target":34963}],"materials":[{"name":"mat","pbrMetallicRoughness":{"baseColorFactor":[0.6415686274509804,0.6215686274509804,0.5909803921568627,1.0]}},{"name":"mat","doubleSided":true,"pbrMetallicRoughness":{"baseColorFactor":[0.5,0.5,0.5,1.0]}}],"nodes":[{"matrix":[1.0,0.0,0.0,0.0,0.0,0.0,-1.0,0.0,0.0,1.0,0.0,0.0,0.0,0.0,0.0,1.0],"name":"Z_UP"}],"scene":0,"scenes":[{"nodes":[0]}],"buffers":[{"byteLength":0}]}   

IFC file to reproduce issue:

AR-Demo_Sample_Single_Building_01.zip

Output from the above execution:

AR-Demo_Sample_Single_Building_01.zip

ifcapps commented 6 months ago

I checked the model with -s 5 parameter and it leads to the same result on Windows, with many empty glb files.

The problem is that a certain minimum file content together with an estimated mesh size of the first mesh already exceeds the 5 MByte. The mesh size estimation can certainly be improved, and if max size exceeded, then shuffling meshes around. But for now I implemented a different fix, by setting the max file size to at least 10 MByte. Implemented in release 3.0.20

markoleek commented 6 months ago

This also happens with s parameter larger than 10 depending on the model. Can share the model privately.

ifcapps commented 6 months ago

Yes, 10 is not enough. I just changed it to 20

xeolabs commented 6 months ago

This also happens with s parameter larger than 10 depending on the model. Can share the model privately.

Yes could you please share the mode via Jira and I'll run it through the converter test suite.

markoleek commented 6 months ago

See here, for some reason the same Jira thread does not contain it. It is reproducible when size <17 https://app.asana.com/0/0/1206440755192332/1206611755032826/f

xeolabs commented 6 months ago

Perhaps ifc2gltf should default and clamp to a minimum tile size of 20, at least as an interim fix until mesh size estimation can be improved?

ifcapps commented 6 months ago

tile size or file size?

ifcapps commented 6 months ago

Minimum file size of 20 MByte is implemented in latest version. File size splitting was introduced because of the 1000 MByte memory limitation on node.js. It was never intended or tested for values below 500 or so.

ifcapps commented 6 months ago

Should be fixed with version 3.0.21 Please re-open if issue persists

markoleek commented 6 months ago

Still happens in 3.0.22 beta with the same model and part size 10 when using Ubuntu.

root@bd7c9e13fdc4:/downloads/Ifc2gltfcxconverter-3.0.22/linux/bin# ./ifc2gltfcxconverter -i Kroon_AR_TP.ifc  -o mudel.glb -m mudel.json -s 10

__ IFC to GLTF converter __ version: 3.0.22 ____________
-s parameter: adjusting max file size to 20 MByte
Loading input config file: ifc2gltfcxconverter.json
Loading file "Kroon_AR_TP.ifc": 100%
Converting IFC geometry: 100%
Exporting to gltf: 100%
Accessors: 2, reused: 0, compression: 0%
Exporting to gltf: 100%
Accessors: 0, reused: 0
Exporting to gltf: 100%
Accessors: 0, reused: 0
Exporting to gltf: 100%
Accessors: 0, reused: 0
Exporting to gltf: 100%
Accessors: 0, reused: 0
Exporting to gltf: 100%
Accessors: 0, reused: 0
Exporting to gltf: 100%
Accessors: 0, reused: 0
Exporting to gltf: 100%
Accessors: 0, reused: 0
Exporting to gltf: 100%
Accessors: 0, reused: 0
Exporting to gltf: 100%
Accessors: 0, reused: 0
...
ifcapps commented 6 months ago

Ok, I will implement some additional checkings for this. Also probably re-write some code. The file splitting is more complex than it seems. Some elements have to be exported repeatedly, with child elements, but without meshes that have already been exported. This is because a valid tree has to be in each file. Metadata and IFC models add their unique complexities as well.

ifcapps commented 6 months ago

A fix is implemented now, version 3.0.23

I added a more detailed explanation about file splitting in the pdf handbook: Splitting the files into several files like this requires an estimate of the gltf and/or .bin file, or glb file. The estimate is based on internal gltf (JSON) data, as well as binary buffers. An exact estimate would slow the process down too much. So the file size will never be 100% accurate. It is recommended to keep the file size limit as high as possible, for example 500 MByte, or 100 MByte, because having more files means that compression (re-use of existing buffers) doesn’t work as effectively any more. The converter detects buffers and accessors with the same content, which happens very often in IFC models, for example the mesh of a door could be repeated thousands of times in a model. Typically, the compression that the converter can achieve is around 50% to 80%. So if the model is split into small files, the overall file size can increase up to 5 times.

ifcapps commented 5 months ago

Solved meanwhile