alicevision / Meshroom

3D Reconstruction Software
http://alicevision.org
Other
11.2k stars 1.09k forks source link

Densified pointcloud is written in insufficient precision with ConvertSfMFormat #2270

Open jsammer-ode opened 11 months ago

jsammer-ode commented 11 months ago

When using the ConvertSfmFormat node to export the densified pointcloud to an .ply file, the points are stored in a low precision (the point coordinates seem to be cut-off at the third comma value). In most cases, this is not a problem because the precision is sufficient when using Meshroom's default local coordinate system. But when using global world coordinates, more comma values are necessary.

A similar problem was already discussed and resolved in https://github.com/alicevision/Meshroom/issues/2103 and https://github.com/alicevision/AliceVision/pull/1486, but only for the SfMTransform node.

The problem would be resolved if there was an option to set the precision in ConvertSfMFormat.

To Reproduce Steps to reproduce the behavior:

  1. Use imagery with GPS locations to generate a mesh.
  2. Adapt the 3D reconstruction pipeline like so: grafik

Here are the settings for the SfMTransform node and the two ConvertSfMFormat node: grafik grafik grafik

Expected behavior The output of ConvertSfMFormat should be the same as the densified pointcloud - it should only be shifted/rotated/scaled. Instead, the pointcloud looks something like this: grafik The correct shape of the pointcloud is preserved if SfMTransform is only used for scaling/rotation, but not if translation is set.

These are the coordinates of the landmarks, where the coordinates seem to be cut-off after a few digits: grafik

I am using Meshroom pre-built, v2023.3.0, on win10

Workaround The log of the SfMTransform node supplies the seemingly correct transformation parameters for shifting the local coordinates to a global system. I am currently using these parameters to apply the coordinate transformatio to a global system myself: grafik

rossoe commented 10 months ago

I'm seeing the same when testing using this barn data

Original XYZ figures using ConvertSfMFormat with no prior SfMtransform: image image

Resulting XYZ figures after ConvertSfMFormat with prior SfMTransform (Translation ticked): image image Log from above SfMTransform node: image

Resulting XYZ figures after ConvertSfMFormat with prior SfMTransform (Translation unticked): image image

Example of GPS data in EXIF on one of the images: image

Checking coords in https://www.geoplaner.com/ to have reference for both geodetic and projected in various formats: image

@jsammer-ode - could you please confirm how you are using the transformation parameters from the log of the SfMTransform to convert original XYZ figures into global world coordinates?

After the SfMTransform I'm converting to ply format in meshroom and then ultimately getting it converted into a pointcloud format which something like QGIS can consume once I can figure out adding a proper georeference procedure.

jsammer-ode commented 10 months ago

Hey @rossoe , I have written my own python script, which uses the rotation/translation/scaling parameters from the log file of the SfMTransfrom node. I have used open3d for it, but I had to come up with some workarounds. For example, if you want float64 precision, you must use the open3d.t.geometry module (instead of the default open3d.geometry. Also, the mesh was mirrored compared to the pointcloud itself, so I had to apply some additional rotation. I was also not able to export a .obj file while preserving the float64 precision, so I am currently exporting the mesh as .ply as well. There might be some python library which is better suited for this problem, but I can confirm that it is possible to use the parameters from the SfMTransform node to transform your pointcloud/mesh to world coordinates.

rossoe commented 10 months ago

Thanks @jsammer-ode would you kindly show the details of the EXIF GPS figures for a single image on the dataset you mentioned above? so I can figure out the maths - re their correlation to the translation figures seen on your screenshot: image

It would be much appreciated if you could also show the first line from your ply file with figures after your python script has converted the translation figures? example of figures from my ply: image

jsammer-ode commented 10 months ago

I don't have that data anymore, so I can't show you. But regarding the math, you want to apply a Helmert Transformation. The translation is merely an offset to the Meshroom-native pointcloud you have received. Your pointcloud is then in a geocentric coordinate system (epsg 4978). Hope this helps