Strange rendering of gocesiumtiler tiles with custom Cesium JS shader

[daz]

I am rendering a point cloud in Cesium JS with a custom shader, it’s displaying perfectly in the sandcastle with the demo point cloud, and with the PNTS tiles that are automatically exported out of DJI Terra, but if I generate tiles from the LAS with gocesiumtiler, it renders fine and in-position with the standard shader/style as simple RGB, but renders complete noise with the custom shader.

The end goal is to change colour based on height in Cesium JS (I had no luck styling based on POSITION_ABSOLUTE so I looked to custom shaders)

Is there something wrong with my gocesiumtiler settings, or is this a bug? Can you offer any improvements to my gocesiumtiler parameters for a 3GB LAS?

Sandcastle with custom shader: ✅

DJI Terra generated tiles: ✅

Same dataset exported as LAS from DJI Terra and tiled with gocesiumtiler: 👎

gocesiumtiler file \
  --epsg 32756 \
  --resolution 1 \
  --depth 8 \
  --min-points-per-tile 5000 \
  --out "tiles-gocesiumtiler" \
  "my-point-cloud.las"

Note that the positions of both tilesets are the same, I have used this method of generating tiles for quite a while but I’ve only ran into problems now that I am using a custom shader.

[mfbonfigli]

I am not quite sure what is causing it considering that from what you say the actual position of the points in space is correct.

Would it be possible for you to share with me, even privately at my email address, the dataset you are using (or even just a small subset) for me to investigate further?

It would also help if you could attach an example of a "working" tileset for reference.

[daz]

No worries, I've just email you a link with the dataset.

[mfbonfigli]

Thanks a lot. I believe that the issue here is that gocesiumtiler never uses local coordinate reference systems. All coordinates are stored in the Cesium default CRS EPSG 4978 (ECEF, cartesian geocentric reference system). Apparently the shader receives the raw coordinates prior to the global transforms, hence your shader will receive EPSG 4978 coords when reading the gocesiumtiler artifact, and coordinates in a local CRS when reading Terra models, hence the math is not going to be the same.

This means that you can't simply subtract the Z of a point to a baseline, as the Z axis is not "UP" relative to your model.

I managed to get a better visualization (not artifact free but almost) by rotating the points so that the Z is "up":

const float GRADIENT_HEIGHT = 30.0;

vec3 hsv2rgb(vec3 c) {
    vec4 K = vec4(1.0, 2.0 / 3.0, 1.0 / 3.0, 3.0);
    vec3 p = abs(fract(c.xxx + K.xyz) * 6.0 - K.www);
    return c.z * mix(K.xxx, clamp(p - K.xxx, 0.0, 1.0), c.y);
}

vec3 applyTransformation(mat4 transform, vec3 point) {
    // Convert the 3D point to a 4D point (homogeneous coordinates)
    vec4 point4D = vec4(point, 1.0);

    // Multiply the transformation matrix by the 4D point
    vec4 transformedPoint4D = transform * point4D;

    // Convert the 4D result back to a 3D point by dividing by w (in case w is not 1)
    return transformedPoint4D.xyz / transformedPoint4D.w;
}

void vertexMain(VertexInput vsInput, inout czm_modelVertexOutput vsOutput) {
 // vec3 positionMC = rotateToZAxis(vsInput.attributes.positionMC);

  mat4 rotationMatrix = mat4(
    -0.479541502619441,-0.8775192005110137,0.0,0.0,
    -0.35640222084710366,0.1947645777122581,0.9138075378567514,0.0,
    -0.8018836600409944,0.43820863980879826,-0.40614749015127727,0.0,
    -5117458.849367955,2796558.644859263,-2574599.756146693,1.0
  );
  vec3 positionMC = applyTransformation(inverse(rotationMatrix), vsInput.attributes.positionMC);

  vec3 referencePointMC = vec3(0.0, 0.0, 50 );

  float distance = abs(referencePointMC.z - positionMC.z);

  float hue = (mod(distance, GRADIENT_HEIGHT)) / GRADIENT_HEIGHT;
  vec3 rgb = hsv2rgb(vec3(1.0 - hue, 1.0, 1.0));
  v_display_mode_styling_color = rgb;

  vsOutput.pointSize = 2.0;
}

Note that I took the transform matrix from the terra tileset.json you sent me.

Unfortunately this is hard to fix on your side. You could tweak with the transform, but this also means that you need to store the transform matrix for each model somewhere or figure out a way to compute it dynamically.

I think this is a good use case. I will consider changing the tiler to store the points using a local "Z-up" reference with a global transform, this should help with these kind of tasks. I'm not sure on the effort required on my side to implement this change though but I believe it might be quite a lot.

[daz]

Thanks for putting the work in. I'd be happy to manually store the transformation for each tileset, providing I can get it perfect, but I was still getting artifacts with the rendering; It’s sort of placing triangle tiles rather than the squares that I’m used to getting out of ContextCapture, And the tiles are showing different colours in places they shouldn’t be. Is there something I could mitigate by tweaking the transformation matrix? I’ve attached pictures so the community can see the results.

I would just use Terra tileset but I need to edit the LAS prior to tiling. Any other ideas? Changing the Coordinate system that Terra exports the LAS? Converting the Coordinate system of the LAS?

DJI Terra tiles:

Applying mfbonfigli’s transformation in the shader to the gocesiumtiler tiles, same data set but using the LAS from Terra: