KhronosGroup / glTF

glTF – Runtime 3D Asset Delivery
Other
7.2k stars 1.14k forks source link

Why the 32 bit limit on GLB file size? #1438

Open anderejd opened 6 years ago

anderejd commented 6 years ago

This unnecessarily limits the scope of the format. What are the reasons behind the ~4 GB limit? Are there any plans to lift this to 64 bit in a future version of the specification?

https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#binary-gltf-layout

length is the total length of the Binary glTF, including Header and all Chunks, in bytes.

bghgary commented 6 years ago

I don't have answers for the questions, but for context, can you list some use cases for a >4GB GLB file?

lexaknyazev commented 6 years ago

Those could probably be extreme high-density point clouds or extensive use of ultra high definition floating point textures. Anyway, the limit affects only GLB container format - a glTF asset with binary data in separate file(s) has no implicit limit on data size.

anderejd commented 6 years ago

can you list some use cases for a >4GB GLB file?

I have many times worked with scenes containing above 4 GB of only geometry, mainly offline rendered car visualizations where the tesselation must be very high to get good results in print resolution. That example may not be what GLB is designed to handle, but it could handle that kind of scene size, if the uint32 sizes were upgraded to 64 bit.

Another use case I can think of is transferring very-high resolution models between different automated tools in a custom content creation workflow.

a glTF asset with binary data in separate file(s) has no implicit limit on data size.

That'a good to know, thank you, I failed to find that information in the specification.

To clarify, in the following example, do the bufferLength property allow unsigned 64 bit integers?

{
   "buffers": [
       {
           "byteLength": 102040,
           "uri": "duck.bin"
       }
   ]
}
lexaknyazev commented 6 years ago

As a general guiding rule, glTF should be seen as a "last mile" transmission format (like JPG for images) as opposed to authoring/interchange formats like USD.

I failed to find that information in the specification.

JSON-Schema for buffer.byteLength property defines only minimum value, so glTF spec doesn't limit maximum buffer size. Common JSON parsers should correctly handle integer values at least up to 253.

anderejd commented 6 years ago

JSON-Schema for buffer.byteLength property defines only minimum value, so glTF spec doesn't limit maximum buffer size. Common JSON parsers should correctly handle integer values at least up to 2^53

That answers my question about external buffer size limits, thanks! That was my hope and expectation since that's the javascript integer max, and gltf 2.0 seems to have quite a few js based viewers.

zellski commented 6 years ago

I think it makes good sense to keep glTF itself limitless, but to accept 4 GB as a reasonable upper limit on a single .GLB file. While self-contained files are incredibly convenient for many use cases, once we're dealing with assets in the 10 GB range we've arguably left that degree of convenience behind, while at the same time, it becomes increasingly meaningful to be able to stream individual components of the assets, such as huge textures or geometry LoD buffers via distinct URIs. The exploded, explicit version of the format is fine here. Maybe by GLB 3.0, the landscape will have changed...

anderejd commented 6 years ago

but to accept 4 GB as a reasonable upper limit on a single .GLB file

This artificial limit makes absolutely no sense, not even decade ago. This is an obvious artificial limit, please consider fixing it.

Maybe by GLB 3.0, the landscape will have changed...

This is what I'm trying to ask for. Please consider fixing this for the 3.0 version. Until then it's reasonable, for my use case, to avoid the GLB format altogether and work with external buffer files, since I know that I am going to hit this limit sooner or later.