GLTFLoader: HTTP Range Requests support

takahirox commented 2 years ago

Is your feature request related to a problem? Please describe.

GLTFLoader first loads the entire bundle glb file and then starts the parse. It is inefficient for network usage if users want to first partially load the assets and then lazily load the others on demand (eg. glTF LOD extension).

Describe the solution you'd like

Support HTTP Range Requests in GLTFLoader.

Changes probably needed:

FileLoader supports HTTP Range Requests #24485
First load only the glb header
Load partial buffer for a buffer view with HTTP Range Requests

Additional context

With the changes, the number of requests can be many. How will the response time be? Longer because of many requests? Or shorter because of the parallelism?

donmccurdy commented 2 years ago

Sorry for the late reply here — please see my response in https://github.com/mrdoob/three.js/pull/24580#issuecomment-1235649799. Thanks for opening this discussion!

takahirox commented 2 years ago

Hi Don @donmccurdy , thanks for the comments at https://github.com/mrdoob/three.js/pull/24580#issuecomment-1235649799

I will reply here in this thread because most of the comments look GLTFLoader stuffs, not FileLoader stuffs.

if you're using LODs, do you often want separate vertex streams for each LOD?

I may choose separate vertex streams for each LOD if an application firsts loads the lowest levels and then progressively loads the other levels on demand. The response time can be much shorter because the assets can be visible when the lowest levels are ready. If a big vertex stream is shared, need to wait for the big vertex stream download completion. Similarly I may choose separate smaller textures for lower level materials.

for progressive loading what do you think of using range requests vs. partitioning data into multiple .bin files?

I don't have a quantitative number, but I think a single bundled glb is one of the popular format. Actually I saw some web applications supporting only bundled glb (perhaps because of no need of thinking of archive).

I think it is good to enable to get the benefit from partial and progressive loading even for bundled glb.

Babylon.js supports glb progressive loading + LOD + HTTP range requests. https://doc.babylonjs.com/divingDeeper/importers/glTF/progressiveglTFLoad I think supporting the same feature in Three.js would be good.

It is just all a tradeoff in terms of complexity

Yes, agreed with it's a tradeoff. But personally I don't think it adds a big complexity in the codebase.

This is a code example

https://github.com/mrdoob/three.js/compare/dev...takahirox:three.js:RangedRequestGLTFLoader

The basic changes are

Move the current Binary extension handler to glb loader supporting content load and buffer/buffer view load from its bin chunk
Add a glb progressive loader having the same interface with the glb loader above
Check if a file is glb at at the begining .load() and use the glb progressive loader if it's glb

If the changes are not acceptable I want to think of achieving the progressive loading with an external plugin by using the plugin system (some refacroing might be needed in the loader tho) because Mozilla Hubs needs this feature.

elalish commented 2 years ago

In addition to LODs (there's a related spec discussion happening around 3D tiles for glXF, which would accomplish this via separate files rather than range requests) I think these range requests have another powerful motivator: parallel compilation. Right now we have to wait until the entire file is fetched before we start compiling our shaders, however all the info we need to populate the shaders is in the tiny JSON part. If could fetch the JSON first, we could start compiling the shaders while we wait for the geometry and textures to come down. Likewise, we can push each texture to the GPU as soon as it comes down, parallelizing most of that work with the network as well.

donmccurdy commented 2 years ago

@elalish I think the Streams API might be the way to go for what you're describing, less time waiting on request round trips maybe? All the same it's a great illustration.

Sorry if this is a longer thought than really belongs in this thread, but I've been feeling for a while that GLTFLoader is becoming pretty complex. It's 4500 LOC in a single file; its unit tests have been removed, and this puts more burden on manual testing. Pre-compiling shaders, streaming textures, early uploading textures, streaming LODs, etc. all does sound like a good thing for our users. But I'm not sure we're setting ourselves up for success in maintaining these features here.

I've also heard complaints from advanced developers who wish GLTFLoader was more memory-efficient — e.g. fewer callbacks — for loading glTF models into a running application without triggering GC and dropping frames. Doing more work synchronously does conflict with our extension API and lazy-loading/streaming features, though.

Eventually, we may need more than one loader in order to support some of these goals. Probably in a separate repository that can run its own unit tests. I'm not sure where to draw that line, and which features should be scoped for a separate loader, but those needs don't seem too far away.

takahirox commented 2 years ago

Thanks for the comments.

I agree with that GLTFLoader should keep good maintainability. I still don't think adding HTTP Range requests support adds a big complexity but also I don't think we should do it now due to the lack of automatic tests for GLTFLoader.

Instead, I will try to think of adding the extensibility for loading glb (maybe with the plugin API). I want to start with refactoring glb handling.

I hope we can keel this issue opened until we will add such an extensibility (or will give it up).

takahirox commented 2 years ago

I experimented HTTP ranged requests with #24580 and the gltf loader plugin API. I succeeded by using .loadBufferView hook.

Plugin source code: https://github.com/takahirox/three.js/blob/GLTFLoaderRangeRequestPlugin/examples/jsm/loaders/gltf-plugins/GLBRangeRequestsPlugin.js

Demo: https://raw.githack.com/takahirox/three.js/GLTFRangeRequestsDemo/examples/webgl_animation_keyframes.html

There are some limitations, eg. it can't be used with anther plugin using loadBufferView() hook like EXT_meshopt_compression handler, but it's likely possible to support HTTP range requests + glTF loader + bungled glb as a plugin and it's managed in external repository or in user codes.

I would like to keep this issue opened at least until #24580 will land.

mrdoob / three.js

GLTFLoader: HTTP Range Requests support #24506