PDAL prevents LAS/LAZ 1.4 from opening if global_encoding set as 1

lgouzin commented 1 year ago

What is the bug or the crash?

In QGIS 3.26, 3.28, some point-cloud files in LAS or LAZ 1.4 can not get loaded. This seems to be caused by PDAL preventing to read when global_encoding is set to 1

Steps to reproduce the issue

Load a LAS/LAZ 1.4 with Global encoding set to 1. The file does not load and returns an error message. The same file loads fine in other software.

Versions

QGIS version 3.28.3-Firenze QGIS code revision c12bcb2f76c Qt version 5.15.3 Python version 3.9.5 GDAL/OGR version 3.6.2 PROJ version 9.1.1 EPSG Registry database version v10.076 (2022-08-31) GEOS version 3.11.1-CAPI-1.17.1 SQLite version 3.39.4 PDAL version 2.4.3 PostgreSQL client version unknown SpatiaLite version 5.0.1 QWT version 6.1.6 QScintilla2 version 2.13.1 OS version Windows 10 Version 2009

Active Python plugins autoSaver 2.6 LAStools 1.4 nominatim 1.4.2 ntv2_transformations 0.20 OSMDownloader 1.0.3 plugin_reloader 0.9.1 pointsamplingtool 0.5.3 qchainage 3.0.1 QuickOSM 2.0.0 quick_map_services 0.19.31 Serval 3.10.2 slyr_community 4.0.6 db_manager 0.1.20 grassprovider 2.12.99 MetaSearch 0.3.6 processing 2.12.99 sagaprovider 2.12.99

Supported QGIS version

[ ] I'm running a supported QGIS version according to the roadmap.

New profile

[ ] I tried with a new QGIS profile

Additional context

No response

roya0045 commented 1 year ago

Is it qgis specific or should this be reported directly in PDAL instead?

lgouzin commented 1 year ago

Good point. PDAL info doesn't read it either. PDAL: readers.las: Global encoding WKT flag not set for point format 6 - 10. It shows as fixed though https://github.com/PDAL/PDAL/issues/3803 https://github.com/PDAL/PDAL/pull/3818

uclaros commented 6 months ago

@lgouzin can you share such a file that fails to load?

lgouzin commented 6 months ago

@uclaros sure, here is a LAZ: https://drive.google.com/file/d/17VvGIpInvSQk1sSlA6kOBzQ5ppHUJQyW/view?usp=sharing

sepa-f1 commented 5 months ago

I have the same issue and whatever I tried I couldn't make it work. I could add LAZ to layer in one of the previous QGIS versions but since the upgrade it stops with error. PDAL is 2.6.0 version and that should have it fixed...

Anyone know exactly where to put the readers.las.nosrs=true parameter that Hobu mentions in https://github.com/PDAL/PDAL/issues/3803?

uclaros commented 4 months ago

I'd argue this is not a bug, rather a strict implementation of the laz spec by pdal.

Global encoding is a bit field, comprising of 16 individual bits, 5 of which are actually used:	bit	usage
bit0	GPS time type
bit1	Waveform Data Packets Internal
bit2	Waveform Data Packets External
bit3	Synthetic Return Numbers
bit4	WKT
bit5-bit15	Must be zero

Point formats 6-10 (laz 1.4) require by the laz spec a WKT srs definition which also means bit 4 to be set to 1.

Your las file properly includes a WKT srs definition but the global encoding value is bad, as it is `1` in decimal for the whole value, ie `0x01` in hex or `0b00000001` which translates to the following bits:	bit	usage
1	GPS time type
0	Waveform Data Packets Internal
0	Waveform Data Packets External
0	Synthetic Return Numbers
0	WKT

making it essentially invalid according to the laz spec, which is why pdal also doesn't read it.

I believe the intention there was to have only the wkt bit set, ie 0b00010000 (0x10 in hex or 16 in dec) or both the gps time type and wkt bits, ie 0b00010001 (0x11 in hex or 17 in dec)

lgouzin commented 4 months ago

That's right, global_encoding should not be 1 in this dataset. I believe it is still possible to display the data without forcing the end-user to reprocess their LAZ files? Other software can display the data. And GPS time can be figured out between week seconds or standard/adjusted GPS time. Maybe something a bit like a vector layer with a funny ESRI SRID without EPSG. The data get displayed anyway and show 'unknown', leaving the user the choice to read the SRID description and then assign another one if needed.

wonder-sk commented 4 weeks ago

A quick summary of the current state:

in older PDAL (versions < 2.4) this was just a warning
in PDAL >= 2.4 and < 2.7.2 this is a fatal error
in PDAL >= 2.7.2 (out soon) this will be again a warning only (see https://github.com/PDAL/PDAL/pull/4414)

In QGIS 3.38, the "untwine" command that indexes point cloud files has a new --no_srs argument that will cause the LAS reader to ignore reading of file's CRS, and thus ignore this fatal error. The QGIS application does not pass this argument though (because it would ignore CRS even when CRS is valid), so it is only possible to do this from command line.

The proper solution will be to upgrade to PDAL >= 2.7.2 in the QGIS installers.

qgis / QGIS