Model: interpolate per-vertex feature IDs

[ptrgags]

According to the EXT_mesh_features spec,

Implementation note: For a primitive with feature ID attributes, points in the interior of a triangle or line segment should be considered to belong to the feature associated with the nearest vertex.

Conceptually, that looks like this:

If we want to implement this here in WebGL, it would take a couple extra attributes to do this in the fragment shader: (below describes the case for triangles, lines would be similar but only 2 vertices)

• barycentric coordinates - vec3 barycentric = (a, b, c) or a vec2 given that a + b + c = 1 by definition
• uninterpolated feature IDs - vec3 (featureIdA, featureIdB, featureIdC) per feature ID set. All three vertices would get all three feature IDs. So for the diagram above, all three vertices would receive the value vec3(2, 4, 5).

This feels a bit wasteful, but in the fragment shader there's no way to get the original vertex values (at least as far as I know?).

This is not needed for per-face metadata with gl.TRIANGLES or per-edge for gl.LINES (as you can set all feature IDs to the same value), but in any other case you'd need the more general "nearest vertex" interpolation.

---

All this said, It would be best to see how often this is needed before implementing this part of the spec.

[javagl]

A small example data set for basic tests:

9935 per-vertex feature IDs.zip

The archive contains

• A tileset with a single glTF that contains the feature IDs in a form that is required for this issue
• A sandcastle for basic tests
• Some information about the feature ID structure

The latter is just a summary of the feature IDs that are assigned to the vertices of the four quads:

        int featureIds[] =
        {
            0, 0, 0, 1, // Lower left
            2, 2, 3, 3, // Lower right
            4, 5, 5, 5, // Upper left
            6, 7, 8, 9, // Upper right
        };

The custom shader of the sandcastle just assigns colors (basically a binary representation of the IDs), as in

        if (id == 0) {
          color = vec3(0.0, 0.0, 0.0);
        } else if (id == 1) {
          color = vec3(0.0, 0.0, 1.0);
        } else if (id == 2) {
          color = vec3(0.0, 1.0, 0.0);
       ...

This currently looks like this:

Implementing a "proper" visualization of the IDs based on the barycentric coordinates (essentially creating a 3-point Voronoi diagram) may require some tricky shader magic and/or additional ("artificial") vertex attributes.

[gkjohnson]

Just jumping in with the details of how this can be rendered since I've been working on a mesh features implementation. I'll respond to #764 later on once I have a more full demo available. But here's an example of what's required in order to render ids "properly" based on closest vertex - sorry it's in three.js:

codesandbox link

And some explanations of the code needed to prepare the geometry:

// "mesh" is the model with feature Ids from the gltf

// de-index the geometry
const geometry = mesh.geometry.toNonIndexed();
const vertexCount = geometry.attributes.position.count;
mesh.geometry = geometry;

// initialize the barycentric coordinate attribute -
// alternatively these could be output from the fragment shader based on gl_VertexID
const barycoordAttr = new THREE.BufferAttribute(new Float32Array(vertexCount * 3), 3);
geometry.setAttribute("barycoord", barycoordAttr);
for (let i = 0, l = vertexCount; i < l; i += 3) {
  barycoordAttr.setX(i + 0, 1);
  barycoordAttr.setY(i + 1, 1);
  barycoordAttr.setZ(i + 2, 1);
}

// ensure that every vertex has access to all the feature ids used by vertices that
// make up the face.
const featureAttr = geometry.getAttribute("_feature_id_0");
const idAttr = new THREE.BufferAttribute(new Float32Array(vertexCount * 3), 3);
geometry.setAttribute("triIds", idAttr);
for (let i = 0, l = vertexCount; i < l; i += 3) {
  const i0 = i + 0;
  const i1 = i + 1;
  const i2 = i + 2;

  const id0 = featureAttr.getX(i0);
  const id1 = featureAttr.getX(i1);
  const id2 = featureAttr.getX(i2);

  [i0, i1, i2].forEach((index) => {
    idAttr.setXYZ(index, id0, id1, id2);
  });
}

And this is the fragment shader logic for generating ids:

      varying vec3 _ids;
      varying vec3 _barycoord;

      vec3 idToColor( float id ) { /* ... */ }

      void main() {

        int maxIndex = 0;
        if ( _barycoord[ 1 ] > _barycoord[ maxIndex ] ) maxIndex = 1;
        if ( _barycoord[ 2 ] > _barycoord[ maxIndex ] ) maxIndex = 2;

        vec3 col0 = idToColor( _ids.x );
        vec3 col1 = idToColor( _ids.y );
        vec3 col2 = idToColor( _ids.z );

        vec3 arr[ 3 ];
        arr[ 0 ] = col0;
        arr[ 1 ] = col1;
        arr[ 2 ] = col2;

        gl_FragColor = vec4( arr[ maxIndex ], 1 );

      }