gadomski / las-rs

Read and write ASPRS las files, Rust edition.
MIT License
73 stars 32 forks source link

Better performance for LAZ files #64

Closed PaulWagener closed 6 months ago

PaulWagener commented 8 months ago

It is possible to increase the performance of decoding LAZ files by an order of magnitude by using the parallel feature of the laz-rs crate.

I've created an example of this in this commit: https://github.com/PaulWagener/las-rs/commit/a664962ed598a3db39906895320bc01e2242a526

Which on my MacBook M3 gives as output:

cargo run --example performance --features laz --release -- ./51DZ2_05.LAZ
    Finished release [optimized] target(s) in 0.02s
     Running `target/release/examples/performance ./51DZ2_05.LAZ`
Original reader
max_x: 160019.999
Run 1/5: 11.96s
Run 2/5: 12.21s
Run 3/5: 12.36s
Run 4/5: 12.33s
Run 5/5: 12.34s

LAZ parallel reader
max_x: 160019.999
Run 1/5: 1.86s
Run 2/5: 1.84s
Run 3/5: 1.84s
Run 4/5: 1.85s
Run 5/5: 1.84s

Process finished with exit code 0

(the file in this run can be downloaded here)

This does require to load the entire file in memory which is not a great fit for the current API and might not be possible for some workloads.

Is this something to consider for this crate? I'd like to create a pull request but am unsure on how to integrate it.

gadomski commented 8 months ago

Is this something to consider for this crate? I'd like to create a pull request but am unsure on how to integrate it.

Yes, I'm generally in-favor of anything we can do to help I/O, including parallelization and/or async work. PRs welcome, but I apologize because I don't have the bandwidth right now to provide integration suggestions (this is a hobby project for me) šŸ™‡šŸ¼.

tmontaigu commented 6 months ago

I'll take care of this, it should be rather easy

gadomski commented 6 months ago

Closed by #70 and the v0.8.4 release.