Reduce memory overhead

[tomvandig]

• Allow tokenizer to work with streamed data, operating on chunks instead of the full buffer.
• Allow meshes to be queried one by one
• Make mesh-caching optional, discard meshes after querying if caching is disabled
• Improve mesh-caching to only cache what needs to be cached
• Reduce memory fragmentation in geometry generation phase by using a custom allocator for the geometry buffers.

[agviegas]

Hey @tomvandig we are using the issues to solve bugs mainly, so to have this section as clean as possible I'm closing this one (for now 🙂).

[pablo-mayrgundter]

Hi Antonio, has this come up again? It would indeed be very nice to be able to pass a stream to ifcjs for reading, to avoid the memory duplication.

[agviegas]

Hi @pablo-mayrgundter right now we are working in fragment, a new geometric system based on instancing that should reduce memory drastically. In addition, if you want to work with big models, we have a pre-processing workflow that consists of exporting the IFC into glb files (for geometry) and JSON (for data). STEP files were not designed to be efficient, and this conversion allows to work with much bigger models in a more scalable way. You can see an example of both export and import here and here respectively, and the live examples here and here. As you can see, all the tools work with glTF as well.

The idea is that you only would need to make this conversion once and then store it either in a server or locally in the frontend (e.g. using indexedDB). Then, the model loads much faster and the memory consumption is much lower (even including the possibility of dropping the JSON of properties in a proper database). Let us know if you have any questions about this!