Mesh decimation (vtkQuadricDecimation) seems not working correctly on Windows and vcpkg packages

[matlabbe]

Here is an example with same export configs (limiting the mesh to 50 000 polygons). On top is the expected result, on bottom is the buggy result (some surfaces are missing and some vertexes are linked to a lot of polygons). The buggy version takes also a lot of time!

Working configurations:

• PCL 1.11.1 with VTK 8.2 (tested on Mac, homebrew)
• PCL 1.8.1 with VTK 6.2 (tested on Windows with appveyor artifacts)
• PCL 1.11.1-dev (acf10d4a9) with VTK 9.0.1 (96e6fa9b3f) (tested on ubuntu focal)

Not working configuration:

• Latest PCL (to this date) with VTK 9.0.1 (tested on Windows, vcpkg)

We should test it under linux to make sure it is not a DLL/memory problem on Windows. EDIT: tested on linux, see above. So it is maybe a compilation problem on Windows and vcpkg packages, or some wrong dlls taken at runtime causing memory issues between dynamic libraries.

[doublechenching]

I have the same problem when using vtk9.0. Numerical optimization problem? always print "unable factor linear system" last for several minutes

[matlabbe]

The ubuntu version with VTK9 works as expected. The VTK6.2 built manually from source on Windows worked too. I checked VTK and PCL related code, and it seems to be the same for the past years. Maybe vcpkg version added some related patches or have different build options than the default ones when built manually. Rebuilding VTK9 from source outside vcpkg could be a nice test to compare.

[Connor-Bowley]

I have opened a PR into VTK that may fix this issue. I was running into similar decimation output on a different application. It seemed that when using Visual Studio with the toolset 14.32.31326 one of the calls in vtkQuadricDecimation was being incorrectly optimized away.

https://gitlab.kitware.com/vtk/vtk/-/merge_requests/9388