Update Untwine to avoid LAS / LAZ Shuffle in custom attributes

vincentBenet commented 3 months ago

[Linked to Laspy issue: https://github.com/laspy/laspy/issues/311]

What is the bug or the crash?

Custom attributs are shuffle when LAZ file is open by QGIS. File are created using laspy package.

The shuffle seems to depends on the number of attributes, OS and QGIS version.

Steps to reproduce the issue

Create a LAS file using laspy having multiple custom attributes.

def reproduct_issue(path: Path) -> None:
    header = laspy.LasHeader(point_format=3, version="1.4")
    las = laspy.LasData(header)
    las.add_extra_dims(
        [
            laspy.ExtraBytesParams(name="b", type=np.float64),
            laspy.ExtraBytesParams(name="bx", type=np.float64),
            laspy.ExtraBytesParams(name="by", type=np.float64),
            laspy.ExtraBytesParams(name="bz", type=np.float64),
        ],
    )
    las.x = np.linspace(0, 100, 100)
    las.y = np.linspace(0, 100, 100)
    las.z = np.linspace(0, 100, 100)
    las.b = np.full(100, 0, dtype=np.float64)
    las.bx = np.full(100, 1, dtype=np.float64)
    las.by = np.full(100, 2, dtype=np.float64)
    las.bz = np.full(100, 3, dtype=np.float64)
    las.write(str(path), laz_backend=laspy.LazBackend.LazrsParallel)

Open the file using QGIS with drag'n drop
Open a point attributs and see error with shuffle values

By opening theses values with laspy the order is correct:

las = laspy.read(str(path))
>>> las.points[0].b
0.0
>>> las.points[0].bx
1.0
>>> las.points[0].by
2.0
>>> las.points[0].bz
3.0

Versions

QGIS 3.34.5 LTS laspy 2.5.3

Supported QGIS version

[X] I'm running a supported QGIS version according to the roadmap.

New profile

[X] I tried with a new QGIS profile

Additional context

LAS & LAZ files : files.zip

hobu commented 2 months ago

I suspect this might be the Untwine version that's being used. I tested it in

https://github.com/laspy/laspy/issues/311

using the latest Untwine https://github.com/hobuinc/untwine/commit/255d30b0ca6aec8100405b382f66a42fc4e59e1c and it works as expected.

vincentBenet commented 2 months ago

Nice @hobu ! If I understand correctly, you have runned QGIS with modified source code and having untwine version updated. By doing so, the shuffle is not present anymore when you drag'n drop the las file in QGIS layer panel ?

hobu commented 2 months ago

I did not test QGIS at all, just Untwine which is what QGIS uses to create the COPC file when you drag-n-drop data onto it. I suspect the version of QGIS you are using is using an older version of Untwine that may have had issues with extra byte dimensions. I hope one of the QGIS devs can report what Untwine version is currently used by QGIS 3.34.5 LTS.

uclaros commented 2 days ago

Apparently the 3.34 branch is using a version of untwine that is a couple commits before v1.0.0 ... I wouldn't vouch for backporting latest untwine to the lts branch, especially after the changes in classFlags handling.

I'm closing this as it's both fixed upstream and pulled into latest qgis version 3.38

qgis / QGIS