search

connormanning / entwine

Entwine - point cloud organization for massive datasets
https://entwine.io
Other
451 stars 128 forks source link

Performance enhancements #184

Closed connormanning closed 5 years ago

connormanning commented 5 years ago

This PR changes the memory model of Entwine by making the ownership of active nodes more deterministic. It lets us be more efficient about when to serialize/awaken nodes, at the expense of some complexity in a new caching layer. The result should be faster builds with more efficient and uniform output in general.

Independent from these changes is also an important upstream change in PDAL which greatly improves Entwine's performance for LAZ output over long-running builds: PDAL/PDAL#2452. This change was released yesterday in PDAL version 1.9.1, and can make a big difference for builds that run for more than a few minutes. Here's a comparison of the same dataset being built over a long time. It's unitless and meant only to show the compounding effect over long periods: pdal-2452

I'll merge this soon and plan on releasing Entwine 2.1 in a week or so. To test, use the latest connormanning/entwine docker tag, or if building natively, use the PDAL 1.9.1 or master branch.

Potentially resolves #170 and #181.

nicolas-chaulet commented 5 years ago

That ‘s awesome, thanks for the great work!!