zeux / meshoptimizer

Mesh optimization library that makes meshes smaller and faster to render
MIT License
5.52k stars 474 forks source link

simplify: Add color support to simplifyPoints #619

Closed zeux closed 11 months ago

zeux commented 11 months ago

Following the new reservoir code, we can now easily accumulate colors into the reservoirs, and add the color error to the position error with a user-supplied weight. The color error is squared distance; as such, we scale it by the squared weight.

In addition, we now use color alpha as a point weight; the weighted sum will thus bias towards more opaque points.

Note that the alpha is not accounted for during point selection.

zeux commented 11 months ago

A few options for handling alpha:

... one issue with all of these is that no point clouds in my test set have alpha, to the point where I'm not sure we even need to support alpha to begin with. Need to see if I can find better test assets here...

zeux commented 11 months ago

I found a grand total of one Sketchfab asset with alpha :D but for that asset whether we support alpha or not seems completely immaterial as the alpha is relatively uniform with few deviations across the entire cloud, whereas the current weighting and potential improvements to candidate selection only matter when alpha is discontinuous.

There's also a downside wrt current implementation in that it requires 4-float colors; since alpha is rarely useful, this unnecessarily restricts the operation for glTF 3-float colors by requiring to expand the buffer to 4-floats. That said, without alpha the solution feels a little incomplete :-/

zeux commented 11 months ago

Since we're lacking good test cases, and meshopt_simplifyPoints was apparently experimental with no color support and a significant quality bug for 4 years, it's probably fine to just add RGB support for now :) We can add alpha separately in the future, with a few different interface/implementation options to consider.

zeux commented 11 months ago

Motivating example (scene from https://sketchfab.com/3d-models/little-cabin-3b9fbbd468e54227ad7ef7e74f8f479c, simplified with ratio 0.2):

Before: Screenshot from 2023-10-12 18-20-26

After: Screenshot from 2023-10-12 18-20-30

The colors on the simplified result are less noisy and it is more representative of the source scene.