Summary of possible changes for the specification of `3DTILES_implicit_tiling`

There are several smaller possible changes for the 3DTILES_implicit_tiling specification. This is a short summary of these, with cross-links to existing issues, or summaries that may be used to start issues when there is the need for further disussion.

Meaning of "Internal buffer"

The specification referred to the "internal buffer" of a subtree file, without explicitly saying what this is. This is clarified via the ~~(pending)~~ https://github.com/CesiumGS/3d-tiles/pull/615.

Handling of leftover bits in availability bitstreams

^{Via slack}

The specification currently does not say anything about the unused bits of availability bitstreams. Specifically: The value of the last 3 or 7 bits of the last byte are not determined by the availability. The specification should explicitly require these bits to be set to 0. This makes sure that two binary files that carry the same availability information are bitwise equal.

Subtree JSON format

Right now, the subtree file format specifies .subtree files to be binary files, consisting of a JSON chunk and a binary chunk. The JSON chunk may refer to the internal buffer (stored in the binary chunk), or to external buffers (via buffer URIs).

One reason for having "all the data in one file" is that it can reduce the number of requests that clients have to make: They do one request for the .subtree file, instead of two requests for the JSON- and binary file.

But it could still make sense to also allow plain JSON files. These files could still link to external buffers via URIs (if necessary). But having a plain JSON file could be reasonable:

It could be used for simple examples, by making the subtree JSON part human-readable (and human-editable)
It could be used in real-world examples as well, when there are no buffers - namely, when all the availabilities are constant. (This does not seem to be sooo uncommon...)

^{This would also resemble the approach of glTF, with the idea of having a plain JSON file (.glTF) with external references, or a binary file (.GLB) that is (usually supposed to be) self-contained.}

Upsampling requirements based on availability information

A rough summary of this question is: What should a client do when there is an implicit quadtree tile that is neither a leaf nor a tile that has 4 children?

A rough summary of the possibilities seem to be to show the child tiles that are available and...

omit the child tiles that are not available and show "holes" there
or upsample the geometry from the tile to fill the holes that are caused by unavailable children

Alternatively, there could be stricter requirements for the availablility (basically disallowing the case of only having 1,2,3 children). Further details can be found at https://github.com/CesiumGS/3d-tiles/issues/489#issuecomment-1011472667 .

Consistent naming for "number of levels" in trees

^{Via slack}

The "size of a tree" is represented with different terms:

maximumLevel for the whole implicit tree, where maximumLevel+1 yields the number of levels
subtreeLevels for the subtrees, where subtreeLevels-1 yields the maximum level in a subtree

For consistency, the terms could be

maximumLevel and maximumSubtreeLevel or
availableLevels and subtreeLevels (this seemed to be the preferred solution)

This will affect only small parts of the spec, but will affect all existing data sets, and the implementations. (At least, if one wants the variable names to match the properties - i.e. it may imply a "Replace all maxmimumLevel with availableLevels", with a diligent review to get the +1's right...)

Fold extensions into implicit tiling spec

This is suggested at https://github.com/CesiumGS/3d-tiles/pull/549#issuecomment-981995210. It looks like the 3DTILES_metadata extension should become an integral part of 3DTILES_implicit_tiling, so that per-tile metadata can be added to the implicit tileset without explicitly declaring it as an instance of the 3DTILES_metadata extension.

I haven't yet wrapped my head around that - so correct me if I'm wrong. I think it might raise some questions in terms of composition of the JSON schema. For example: The proposed tileMetadata has to be optional, but still refer to the tile.3DTILES_metadata.schema.json to define its type ... where the latter disallows itself to be part of implicit tiling. Or for short: Some technical details have to be sorted out here...

Differentiation between "tile metadata" and "content metadata"

Right now, metadata may be assigned to tiles, but not to tile content. (The latter would only be possible with a quirky workaround of defining single-element groups, and assigning group metadata to them...). Changes in the metadata spec here may affect the implicit tiling spec.

Further issues that are related to implicit tiling, from quickly skimming over the list:

May need some discussion:

https://github.com/CesiumGS/3d-tiles/issues/582 - This may affect the producers and consumers of implicit tilesets, but the exact implications for both of them may still have to be identified
https://github.com/CesiumGS/3d-tiles/issues/569 - Sounds interesting, but ... "lots of details to work out". (I haven't understood how a "tiling scheme" should manifest itself on client side generically...)
https://github.com/CesiumGS/3d-tiles/issues/598 - There are some proposals, but no decision yet

Some of these may be worth talking about, but they will likely not change the spec:

https://github.com/CesiumGS/3d-tiles/issues/576 - This will probably not affect the spec itself, but might become an implementation note
https://github.com/CesiumGS/3d-tiles/issues/547 - Request for clarification for the images (with a rough suggestion of what this might look like)
https://github.com/CesiumGS/3d-tiles/issues/507 - This has been generalized to other operations that clients will have to implement in order to navigate in the tileset. Carving out some "Implementation Note" code snippets from existing implementations (e.g. in CesiumJS and cesium-native), and presenting them in a common, sensibly structured and readable form may be a bit of effort, but could be appreciated by other implementors.

Some notes/updates from an internal discussion:

Handling of leftover bits in availability bitstreams

This was generally agreed on. It might be added in the implicit tiling spec only, or it may be added as a general requirement for binary metadata encoding at https://github.com/CesiumGS/3d-tiles/blob/main/specification/Metadata/README.md#booleans

Consistent naming for "number of levels" in trees

There are some possible interdependencies to the "Upsampling requirements ..." point. Depending on the solution for the latter, the concept of the maximumLevel might be split into two fields, indicating the "maximum level of tiles" and the "maximum level of content", but the details have to be sorted out here. Regardless of that, changing the maximumLevel to availableLevels can already be done (even if the latter is later split into availableLevelsOfTiles and availableLevelsOfContent). The general goal would be to have a variable that represents the height of the tree, akin to the subtreeLevels.

While this will only affect a small part of the specification itself, it may require non-trivial changes in CesiumJS and cesium-native, if the implementations are supposed to resemble the specification at the level of property/variable names.

Fold extensions into implicit tiling spec

This was basically agreed upon. When applying it, one has to take care to

link the JSON schemas properly, so that the type of the tileMetatata (as suggested in https://github.com/CesiumGS/3d-tiles/pull/549#issuecomment-981995210 ) is actually defined via a $ref to https://github.com/CesiumGS/3d-tiles/blob/main/extensions/3DTILES_metadata/schema/tile.3DTILES_metadata.schema.json
note (in the extension specification text) that there is a dependency of the extensions: When declaring the extension as a "used extension" in a tileset, then 3DTILES_metadata also has to be declared as a "used extension" (because the type of the tileMetadata is a type from 3DTILES_metadata)

The "Upsampling requirements..." point is one of the more difficult one. There seemed to be a slight consensus that requiring the client to perform a certain upsampling may be hard to convey (and specify), and it should probably be the responsibility of the producer to generate "proper" tilesets (and providing upsampled data where necessary). But some details about how this will affect the specification still have to be sorted out.

CesiumGS / 3d-tiles