Tests to boost loading of models

[tmarti]

Hello,

As we briefly discussed before, here I open an issue to expose an idea related to the performance of model loading (GLTF models in my case).

The background

When we load a GLTF model (whose size is 8.6 MB) using the GLTFLoaderPlugin class, following can be observer when we launch a performance profile (on Chrome in this case):

This means that it takes 2.6s to load a quite small model.

Further analysis reveals where the time is spent:

So the main time consuming processes are in viewer/scene/math/buildEdgeIndices.js (54.8 % of the model load time) and the transformAndOctEncodeNormals method in viewer/scene/PerformanceModel/lib/batching/batchingLayer.js (25.4 % of the model load time).

This (for that 8.6 MB GLTF file) means that two mentioned pieces of code take themselves more than 80% of the model loading time.

Just before going to the real point of this issue, let's talk a little bit about the two previous pieces of code.

What is the purpose of viewer/scene/math/buildEdgeIndices.js

Looking at the code, this comment is the key:

// an edge is only rendered if the angle (in degrees) between the face normals of the adjoining faces exceeds this value. default = 1 degree.

So the code (by using some welding algorithm) filters out edges if the two faces that are part of the edge do not have a minimum angle between normals.

This makes all sense, so that when enabling edge rendering coplanar faces do not get their connecting edges drawn.

So the purpose of this code seems to be a cleanup of the edges of the mesh.

What is the purpose of transformAndOctEncodeNormals in viewer/scene/PerformanceModel/lib/batching/batchingLayer.js?

I'm not an expert in 3D graphics, but it seems that it is applying some "octhaedron encoding" on the normals of the faces so that they either behave better when rendered from the GPU (better could be more efficient GPU usage or taking less space in GPU memory (the encoding result are only two components instead of the 3 XYZ components if not encoded)).

So the purpose of this code seems to be preparing the normals of the geometry for the GPU rendering.

The real point of this issue

The previous two analyzed pieces of code are actually a preparation stage of the geometry for having optimal GPU based rendering.

The idea is that those two pieces of code do not do any calculations that depend on the runtime or the interaction between loaded models, so they could be precomputed.

This what (in my case) I have:

(1) by some convenience too that extracts data from IFC models (2) by the GLTFLoaderPlugin class, that loads the GLTF file in-JS-memory (3) corresponds more or less to those two analysed pieces of code (4) (I think) it's a combination of shaders and binded webgl arrays

The point is that the process done by (3) is the one who is taking more than 80% of the time dedicated to model loading in xeokit.

If the following could be done...

... that would mean that xeokit, during 3d visualization runtim would not have to do an as much computation demanding pre-processing stage in order to load the model geometry into the GPU, and would allow the model load time by (around) 80% 😄

What will I try to do

If I time pressure at work allows me to do so, I will try to do steps towards a working prototype with the presented idea.

That would make xeokit have a world class model loading performance 💃

I will try to keep this issue updated with the progress :-)

[tmarti]

Well, it seems this one is getting progress.

On a test setup, the model load (4 or 5 models) time dropped from 15.2s to 2.5s.

Just stay tuned for tomorrow news about this one :)

[tmarti]

It's been a while since the last post to this thread..

... but the proposed idea is fully working now 😄, here goes a brief explanation of what's been done up until now:

Further analysis of the loading performance problem

As it's been explained on the first comment, during loading of the GLTF files some are currently done (end to end process):

(a) reading and parsing the GLTF file into in-memory structures (b) converting those in-memory structures into optimal representation for the GPU (c) compiling those GPU-optimized structured into shaders for high-performance rendering (this is where xeokit outstands over xeogl)

The (b) step implies processing done in multiple levels of the GLTF loading process:

1. in GLTFPerformanceLoader class => buildEdgeIndices is invoked
2. in BatchingLayer class => geometry is transformed and quantized, normals are oct-encoded

As this (b) step processing implied multiple classes, it was not easy to find a way to properly extract it.

BUT... what has been done

The goal was to extract that heavy processing outside of xeokit's GLTF loading process, in such a way that a file type is created (maybe .xeokit file format? 😊) inside which all the pre-proprocessing has been already done.

So this is the structure of the idea, having in mind the new .xeokit format:

(1) Create a conversor plugin from GLTF files to .xeokit files

This has been implemented as a new plugin (GLTFToXeokitExporterPlugin), intended to be invoked from the command line within a node app.

(Small hacks are needed for this to be able to invoked and run from node, like mocking some browser dependencies, but don't worry for this, everything works just fine without needed things like the jsdom or canvas node packages 😸)

This plugin can read a GLTF file and output a .xeokit file, where all the heavy pre-processing is already included in the .xeokit file contents.

The exporter plugin relies on the (unmodified) GLTFPerformanceLoader class to do its loading, so if adjustments are done on the way to read or parse GLTF files (for example supporting draco compression), this will be transparent to the exporter.

The example code (without all the mocking stuff) that does a GLTF => .xeokit conversion is the following (this is actually working code from my tests):

var gltfToXeokitExporter = new GLTFToXeokitExporterPlugin();

gltfToXeokitExporter.load ({
    id: "model 1",
    src: 'file:///tmp/structure.gltf',
    xeokitArrayBufferGenerated: function (arrayBuffer) {
        console.log ("Generated arrayBuffer!");
        fs.writeFileSync ("/tmp/structure.xeokit", new Buffer (arrayBuffer));
        process.exit(0);
    }
});

structure.gltf corresponds to this file.

In order to be able to use an unmodifed GLTFPerformanceLoader, the idea of the exporter plugin is to create a fake (mocked) PerformanceModel that captures all invocations to createGeometry, createMesh and createEntity methods. When the fake PerformanceModel has captured all the data from the the GLTF file, it processed to do the heavy processing and finally compresses and outputs an ArrayBuffer with all the data, which can be directly saved as a .xeokit file.

This way of implementing it possibly means that the same scheme could be used to create exporters from any format already supported by xeokit into .xeokit files (although some refactoring could be needed to extract common logic now used only by this exporter plugin).

(2) Create a loader plugin for .xeokit files

Once the exporter has generated .xeokit files, they need to be loaded.

If loading GLTF files uses this very simple code...

const structure = gltfLoader.load({
    id: "structure",
    src: "./models/gltf/WestRiverSideHospital/structure.gltf",
});

... loading .xeokit files uses this other very simple code (using the new XeokitLoaderPlugin class):

const structure  = xeokitLoader.load ({
    id: "structure",
    xeokit: xeokitArrayBuffer,
});

// where xeokitArrayBuffer is the content of a response to the .xeokit file URL with request.responseType = "arraybuffer"
// TODO: support loading directly from a URL with the `src` parameter

This loader loads the equivalent geometry of the converted GLTF file, but in doing so it already has all the heavy processing done in advance, so those .xeokit files' content is propagated straight away to GPU buffers (see next point (3))

(3) (only changes needed to already existing xeokit files) skipping processing

In the case of loading the contents of a .xeokit file, the loader plugin needs to tell xeokit to avoid doing the same heavy processing twice, because the new file format is already processed.

Only for this reason had some xeokit files (PerformanceModel / BatchingLayer) needed to be modified, in order to skip doing the same transforms / quantizations / oct-encoding is they were already done.

And that's all

Just to give some statistics, the comparison between loading a GLTF or a .xeokit version of structure.gltf are given there:

GLTF file

Size: 27.098 KB
Load time (avg of 5 loads after a warm-up phase of loading it 5 times): 3.363 seconds

xeokit file

Size: 1843 KB
Load time (avg of 5 loads after an warm-up phase of loading it 5 times): 0.538 seconds

Some data

Compression ratio = 1843 / 27098 = 6.8 % => 1/14.7th the original size
Load time ratio = 0.538 / 3.363 = 16% => x6.25 speedup

I will now try to discuss with @xeolabs the best way to integrate the changes, and see if there is some use case that needs to be supported before sending the PR 😄

[Amoki]

This is very interesting!

If two doors share the same geometry, meshes for the two doors are duplicated after buildEdgeIndices ? If so, that would reduce the gain with well optimized GLTF files and increase size of the .xeokit file.

[xeolabs]

@Amoki nope not a problem - the edge representation is generated once per reused geometry, so the edges are reused also.

[tmarti]

So there we go 🚀, here it goes the inital version of the code, although there are still some items on the TODO-list yet

Please refer to changes in the following commit on the foked repo for further details: https://github.com/tmarti/xeokit-sdk/commit/da5932a6caafe6d11b3b3d6978d811caa540030d

Will try to organise the TODO list agreed with @xeolabs during this week, just stay tuned as usual 😄

@Amoki, as @xeolabs tells the edge representation is recycled, but in the case of instanced geometries, the positions and vertex indices are duplicated for each instance of the geometry. The .xeokit file generation&load could be adapted to better support this if that creates a problem e.g. as you say by potentially creating huge .xeokit files.

By the moment, the code is open to suggestions and enhancements, and the PR will not be created until some details are polished, but it is now open for insepction.

For sure there are lots of things to improve in the commit, so be nice and show some mercy to me 😄

[xeolabs]

Thanks @tmarti I'll review over the next couple of days.

Yeah the edges positions and indices will need to be recycled for instanced geometries - in a good model there will be a lot of instancing, so that will add up. But this is a great start.

[xeolabs]

Now implemented in v0.3.0 as XKTLoaderPlugin!