Resolution issue in generation of las point cloud

[Agrobit]

How did you install OpenDroneMap? (Docker, natively, ...)?

Docker

What's your browser and operating system? (Copy/paste the output of https://www.whatismybrowser.com/)

Chrome 99 on Windows 10 64-bit Edition

What is the problem?

Point cloud (.laz, .las, .ply) visualization issue in CloudCompare. The point cloud do not render correctly. I am using the following options (default) inside CloudCompare: I click on “apply all” and then on “yes to all”. If I click “no” on the global shift I get an even worse point cloud visualization. The problem seems to be related to the visualization of the point cloud and not to the processing, since the point cloud dimension (number of points) is correct in comparison to a Pix4D processing of the same dataset.

Given results:

What should be the expected behavior? If this is a feature request, please describe in detail the changes you think should be made to the code, citing files and lines where changes should be made, if possible.

Expected results:

How can we reproduce this? (What steps did you do to trigger the problem? What parameters are you using for processing? If possible please include a copy of your dataset uploaded on Google Drive or Dropbox. Be detailed)

To process the dataset, webODM was used with the following parameters with respect to the default ones (also, different parameters have been tested with no successful results:

feature-quality: ultra, min-num-features: 50000, pc-filter: 0, pc-geometric: true, pc-quality: ultra, pc-sample: 0, resize-to: -1, use-3dmesh: true

The dataset is here: https://drive.google.com/file/d/1t_-nTWEkBFoY_Jah7LnkVKP7hLP0aErV/view?usp=sharing

[pierotofy]

Can you share both point cloud files (Pix4D one and ODM one?)

[smathermather]

I'll do you one better. Here is both the (good) results in the odm_filterpoints folder as well as the corrupted laz file: https://drive.google.com/drive/folders/1FnhB_ChKVbsKGowi-n8aTWmFXsq18_Gu?usp=sharing

[Agrobit]

Thank you smathermather, you anticipated me for few minutes! :D

[smathermather]

If you can share the Pix4D one, that would be interesting to see.

[Agrobit]

Yes, no problem. Here you go:

Pix4D (3D Model template) -> https://drive.google.com/file/d/1S6hdZlULy-qQ4JkMZH_pxwYcr67-7-oj/view?usp=sharing

[pierotofy]

This is Stephen's odm_georeferenced_model.laz in my CloudCompare:

Looks fine. This does not look like a bug in ODM, if anything it might be an issue of CloudCompare?

[pierotofy]

You can also drag & drop the laz into https://plas.io/. It looks fine?

[smathermather]

Zoom in. You'll see aliased points that appear to be a floating point issue.

[Agrobit]

[smathermather]

Relevant conversation: https://community.opendronemap.org/t/webodm-vs-pix4d-point-cloud-in-vineyard-analysis/10853/14?u=smathermather

[pierotofy]

The key issue here is the LAS/LAZ format scale format. By default PDAL uses a scale factor of 0.01 https://pdal.io/stages/writers.las.html to convert PLY --> LAZ, but since the model here is such at small scale, you get the imprecision issue.

Can you share the LAS/LAZ file generated by Pix4D?

[pierotofy]

PDAL supports an "auto" parameter for setting scale/offset, it might be interesting if somebody tried to patch https://github.com/OpenDroneMap/ODM/blob/master/stages/odm_georeferencing.py#L117 to add the scale auto parameters and see if the resulting LAZ file opens correctly then.

[Agrobit]

The key issue here is the LAS/LAZ format scale format. By default PDAL uses a scale factor of 0.01 https://pdal.io/stages/writers.las.html to convert PLY --> LAZ, but since the model here is such at small scale, you get the imprecision issue.

Can you share the LAS/LAZ file generated by Pix4D?

Please find here attached the .laz file as it comes from Pix4D without any modification -> https://drive.google.com/file/d/16-mx8MDW9hRpSVOd8b8FR2rwFEZl_Y3z/view?usp=sharing

[pierotofy]

Thanks!

Yeah Pix4D did a better job here and chose a scale value of 0.001. So we should improve this in ODM.

[smathermather]

Hmm, auto might not be the solution:

[INFO]    running pdal translate -i "/datasets/12716ecb-937a-4c66-9a72-278eb73fc8f0/odm_filterpoints/point_cloud.ply" -o "/datasets/12716ecb-937a-4c66-9a72-278eb73fc8f0/odm_georeferencing/odm_georeferenced_model.laz" ferry transformation --filters.ferry.dimensions="views => UserData" --writers.las.compression="lazip" --filters.transformation.matrix="1 0 0 454918.0 0 1 0 9578284.0 0 0 1 0 0 0 0 1" --writers.las.offset_x=454918.0 --writers.las.offset_y=9578284.0 --writers.las.offset_z=0 --writers.las.scale_x="auto" --writers.las.scale_y="auto" --writers.las.a_srs="+proj=utm +zone=36 +south +datum=WGS84 +units=m +no_defs +type=crs"
(pdal translate writers.las Warning) Auto scale for X requested in stream mode.  Using value of 1.0.
(pdal translate writers.las Warning) Auto scale for Y requested in stream mode.  Using value of 1.0.

I'll try next with a fixed 0.001 value. :smile:

[pierotofy]

You can also pass --nostream to disable streaming mode. I'm not sure of the tradeoff between memory/speed here.

[smathermather]

I'll try next with a fixed 0.001 value.

Welp -- no. I just ran it on the wrong dataset. A massive one. 1.0 isn't a bad value to assume for this dataset. Rerunning on the dataset in question... .

Edit: reran on the correct dataset and it still chose 1.0. I think maybe auto doesn't work. I'll set it to 0.001

[pierotofy]

I think it's probably safe to hardcode 0.001 as the new default for all dimensions (x,y,z); with 32 signed int coordinates should be good to represent up to 2**32 * 0.001 --> 4294967.296 --> ~4294 kilometers (1/15th of the radius of the earth). I think that works for most UAV dataset areas.

[smathermather]

Sounds good. It will affect the compression performance a bit, but hopefully it isn't too bad.

[smathermather]

Looks like setting it in just x and y is problematic: :smile:

[smathermather]

Ahh, that's better:

Ok: pull request here: https://github.com/OpenDroneMap/ODM/pull/1435