unitycoder / UnityPointCloudViewer

Point Cloud Viewer and Tools for Unity
https://assetstore.unity.com/packages/tools/utilities/point-cloud-viewer-and-tools-16019?aid=1101lGti
128 stars 15 forks source link

converted point cloud only has discrete z-values #100

Closed nvoelzow closed 3 years ago

nvoelzow commented 3 years ago

When I convert a .LAS point cloud to v1 bin everything looks ok, but if I convert the same file to v2 or v3 (using either the LazConverter17b or the PointCloud Converter v1.72) the result in Unity looks striped as if all z-coordinates are quantized to some grid (like casting to int accidentally somewhere during conversion) pyramid_quant

unitycoder commented 3 years ago

if possible email me link for that las file? (haven't seen this happening with my las test files).

nvoelzow commented 3 years ago

Unfortunately I can't share the LAS file due to license/NDA restrictions (debugging with 27gb scan files is not much fun anyway...)

I managed to extract just the part shown in the screenshot using lassplit and the result is the same - I've emailed the link to the 12mb archive containing the LAS as well as the generated v1 and v3 files to you

nvoelzow commented 3 years ago

running the v2/v3 demos provided in the package works fine, so it seems to be more of a problem with the conversion or loading of the converted files.

conversion from LAS to v3 was done with this command:

$ ./LasConverter.exe -input=/D/Hotfolder/subset.las -version=3 -gridsize=1 -output=/D/Hotfolder/subset/subset.pcroot

unitycoder commented 3 years ago

ok i see the issue, ill check.

unitycoder commented 3 years ago

ok it was precision issue, standalone converter uses Floats, while the old v1 converter uses Doubles. I'll update the converters and message here when they are up.

unitycoder commented 3 years ago

can try latest version now https://github.com/unitycoder/UnityPointCloudViewer/wiki/Commandline-Tools sources are updated also https://github.com/unitycoder/PointCloudConverter

nvoelzow commented 3 years ago

indeed, for whatever odd reason the model is offset by around 3,300,000 in y direction so there are probably not enough bits left in a float...

pyramid_las

I'll check out the new converter - thank you!

nvoelzow commented 3 years ago

ah, the Link to LasConverter 1.73 under CLI-Tools only points to the PointCloudConverter (which unfortunately crashes when processing the 27gb file)

unitycoder commented 3 years ago

it accepts same commandline params, if run without it opens gui. ill check if can add doubles to the old version.

if nothing else, need to split to multiple las files. (for v3 can batch convert multiple files into one .pcroot) also 27gb is probably too much for v2 too.

nvoelzow commented 3 years ago

hmmm even after splitting the LAS file into more reasonably sized chunks of 10M points and running in batch mode the PointCloudConverter crashes with an OutOfMemoryException at the 9th or 10th input file when memory usage of the process reaches ~2.5gb - I guess I'll build a 64bit version and try again

unitycoder commented 3 years ago

ah the build wasn't 64bit, thats quite likely the issue. new build (and source repo updated): https://github.com/unitycoder/UnityPointCloudViewer/wiki/Commandline-Tools

nvoelzow commented 3 years ago

with a 64bit build it looks good so far, it's still working on that dataset and still hasn't crashed (and halfway through it's taking up 15gb of ram)

nvoelzow commented 3 years ago

new problem: I ran out of disk space when the output size reached 350gb (from the 27gb input file) - looking at the previous smaller sample where the file size did not really increase between the LAS and v3 files I'm wondering what could be wrong...

or do the pct and pct.rgb files get recombined at the end of a batch conversion where the file size should shrink back to normal?

nvoelzow commented 3 years ago

still good news: with the 64bit build I was able to convert the original non-split LAS file directly

unitycoder commented 3 years ago

.pct is XYZ data and .rgb is the color data for that tile, so they dont get combined after conversion.

for that should use packed colors, so it already eliminates the .RGB data file completely, better test with smaller cloud or with -limit param to find good values. (not sure if gridsize < 1 creates issues with packed data though).

how many points your 27gb file has? with 10billion points i think my file size was around +200gb.. (using packed colors mode).

but yeah, some form of compression is quite high in the todo list. (now its raw float data)

nvoelzow commented 3 years ago

so the original LAS is 27gb, this is the output when running it from that single file, resulting in ~18.5gb of pct/pct.rgb files:

Reading file (0/0) : d:/hotfolder/inside_830m_20190218.las (26.3GB) Points: 829955385 Saving 1071 tiles to folder: d:\hotfolder\inside_1m

Saving rootfile: d:\hotfolder\inside_1m\inside.pcroot Total points= 829.95M Done saving v3 : d:\hotfolder\inside_1m\inside.pcroot Skipped 19 nodes with less than 1000 points) Finished! Elapsed: 00h 18m 53s 1853ms

what was really weird was that after splitting the LAS into 83 files with 10M points each (~330mb each) and running the conversion on the folder the output directory was getting larger than 350gb, that's why I was wondering if maybe there was an additional post-processing step that would re-combine the tiles at the end to get back to a reasonable size