Performance stats: `ModelExperimental` vs `Model`

[ptrgags]

Now that the ModelExperimental implementation is pretty far along, we wanted to start measuring its performance compared to the old Model. This issue is to record the results.

Testing Loading Time

To start simple, I performed a test of loading times for a few tilesets to cover some common cases.

• Data Used:
  • CDB San Diego model (EULA. tiled with cdb-to-3d-tiles) - this is a collection of 4 tilesets, a mix of b3dm and i3dm models. This tileset features a lot of textures shared between tiles for the buildings.

• Melbourne Photogrammetry (Modified from City of Melbourne 3D Textured Mesh CC BY 4.0) -- an example of b3dm photogrammetry

• Helsinki Point Cloud (courtesy of Aalto University with support from City of Helsinki https://zenodo.org/record/5578198#.YjoTWBNKiu4) -- an example of pnts point clouds.

• My Sandcastle Setup
  • See #10316 for my basic approach
  • Browser: Chrome 100.0.4896.127 (Official Build) (64-bit)
  • OS: Kubuntu 18.04
  • There were some differences, as some of the local tilesets were gzipped. In the spreadsheet (see Results below) I marked cases where the tilesets were compressed.
  • Sandcastle links
  • I used several Sandcastle links with slight modifications (e.g. changing the enableModelExperimental flag). The spreadsheet includes links for every variation used. The links here are just a representative set.
  • Note that these links require the local built version of Sandcastle:
  • CDB San Diego
  • Melbourne Photogrammetry
  • Helsinki point cloud

Results

All the results can be found in this spreadsheet. For the color-coded cells, green means ModelExperimental was faster, red means ModelExperimental was slower at loading the model. Performance Stats.xlsx

Preview:

Takeaways

• Overall the performance felt comparable, loading times differed by a couple seconds, not tens of seconds or more. That said, in most of the cases, ModelExperimental was slower, so it's worth investigating further.
• the CDB elevation and buildings were two cases where ModelExperimental performed better. This may be due to the fact that these models share textures. GltfLoader has better texture caching than the old Model; no duplicate textures should be uploaded to the GPU. However, profiling memory would be a better way to check if this is true.
• Point clouds were the area where ModelExperimental performed the worse by percent difference. The new PntsLoader does work differently than the old PointCloud class, so it would be good to profile this further.
• Likewise, other cases like the photogrammetry one should be profiled further to see why things are slower.
• It would be good to test memory too, though we'd need to implement #9886 first.

CC @lilleyse @j9liu @IanLilleyT

[JiaoJianing]

Hello~ Anything progress? Now we have some huge 3dtiles data, in a fixed view, the loading time will be longer in version 1.97 than pervious version. Have you investigated the cause of the performance drop? @ptrgags