CesiumGS / 3d-tiles-tools

Apache License 2.0
315 stars 47 forks source link

Decode 3D texture coordinates during upgrade #146

Closed javagl closed 3 months ago

javagl commented 3 months ago

This is structurally similar to https://github.com/CesiumGS/3d-tiles-tools/pull/98 (where "structurally similar" means "largely copy and paste"), with a similar summary:

The exact encoding of the texture coordinates is not yet clear. It just seems to be "one (arbitrary) way" of not storing the VEC2/FLOAT (2 4 = 8 bytes), but instead, storing VEC3/SHORT (3 2 = 6 bytes).

In one of the relevant glTF files, the decoding in the shader was done like this:

const float uvMultiplier = 0.0000305185; // 1/32767
v_texcoord0 = a_texcoord0.xy * uvMultiplier * (a_texcoord0.z+32767.0);

This was for an accessor with VEC3/SHORT.

This PR implements the decoding of VEC3/SHORT (or VEC3/BYTE) texture coordinates into VEC2/FLOAT texture coordinates. From a quick test with one of these B3DM files that contain such glTF 1.0 data, the result can be rendered in CesiumJS.

But... there are some guesses and degrees of freedom.

Some of this still has to be sorted out and confirmed, so I'll just open this a a DRAFT for now.


(NOTE: The state in this PR currently tries to ignore missing BATCHID attributes in B3DM files. This should not be necessary. In the final state, this might be omitted. But there are reasons to assume that this is one quirk that may happen for ~"some old legacy data sets" - details TBD)

lilleyse commented 3 months ago

Does this have a name (like "oct-encoded" for normals)?

Not sure. gltf-pipeline used to compress texture coordinates but from a quick glance it's a different technique: https://github.com/CesiumGS/gltf-pipeline/blob/0d44fb90a4bd9e9e6fba5abce3bb92752f801945/lib/compressTextureCoordinates.js

javagl commented 3 months ago

The fact that there could have been "arbitrary" forms of texture coordinate compression in glTF (with all this arbitrariness being compensated in a custom shader) makes it difficult to implement this generically, in a form that always works.

On the one hand, the specific form of compression that was used here appears to be used in multiple files, but it's probably not widespread and unambiguous enough to commit to one decoding method. On the other hand: When we find a glTF with 3D texture coordinates, and no clue about how they might be encoded, then trying to decode them (like in this PR) cannot "break" anything (because leaving them as a VEC3 will make the asset invalid in any case).

No strong opinion, though. Let's leave it as a 'draft' and see whether we encounter more such models, and notice a pattern...

javagl commented 3 months ago

Split up and cleaned up into https://github.com/CesiumGS/3d-tiles-tools/pull/147 https://github.com/CesiumGS/3d-tiles-tools/pull/148