search

jblindsay / whitebox-tools

An advanced geospatial data analysis platform
https://www.whiteboxgeo.com/
MIT License
956 stars 160 forks source link

LidarJoin scrambling points. #17

Closed Hutch07 closed 5 years ago

Hutch07 commented 6 years ago

I ran a test of LidarJoin using WhiteboxTools Runner on 5 files of total point record size 19753893. Version 1.2. These were non RGB points. The data produced in the output file was scrambled and grouped at the XY, ZX and ZY - see attached png file. I tried just 2 smaller files and returned the same result. I used lastools lasmerge and received the correct merge.

wbtjoin

jblindsay commented 6 years ago

Which version of WhiteboxTools are you using? I made some modifications to the LidarJoin tool in the latest version (v.0.9) that may impact this. May I see what the 'correct image' also looks like, i.e. the LASTools image?

Hutch07 commented 6 years ago

Thank you very much for the tools! This is what I have, Windows 10 and Python version 3.7. I have attached 2 views of the same file. This is lasview but it looks the same on other viewers.

whitebox_tools.exe --version whitebox-tools v0.9.0 by Dr. John B. Lindsay (c) 2017-2018

wbt1 2018_0904 wbt2 2018_0904 wbt3 2018_0904

jblindsay commented 6 years ago

This is very odd. Are you able to get me the source LAS files on which you are applying LidarJoin? I'll need to take a closer look at what is happening. As for the difference in file size between lasmerge and LidarJoin, I believe that LASTools outputs LAS v 1.4 while WhiteboxTools outputs LAS 1.3. That may account for the difference there.

Hutch07 commented 6 years ago

I am going to test just 2 files and I will send you the results.

jblindsay commented 5 years ago

Good news, I just encountered this very same issue when I was trying to process LAS files that had been created by using LASTools' LAS2LAS software. As it turns out LASTools can sometimes create LAS files where the point records have more bytes than the specification mandates. I'm not sure what LASTools is storing in these extra bytes, but either way, WhiteboxTools was previously expecting strict adherence to the LAS specifications. I've now re-written the LAS reader to handle extra bytes (ignoring them but not offsetting the other point data) and this seems to have resolved this issue. I've committed the updated code. As such, I am closing this issue. I will be releasing the next version of WhiteboxTools sometime this week, which will have this bug fix. If you happen to test it and encounter any further problems, please feel free to re-open the issue.

Hutch07 commented 5 years ago

Thank you very much for the reply. If you ever get over to Standish, Michigan, stop on in. I will reload WBT and open some other files. We have several systems creating las/laz files - mostly Riegl systems. I haven't used WhiteBox tools much but will try out the new code.

Thanks again,

Cary Hutchinson Surveying Solutions Inc

On Tue, May 21, 2019 at 1:44 PM John Lindsay notifications@github.com wrote:

Good news, I just encountered this very same issue when I was trying to process LAS files that had been created by using LASTools' LAS2LAS software. As it turns out LASTools can sometimes create LAS files where the point records have more bytes than the specification mandates. I'm not sure what LASTools is storing in these extra bytes, but either way, WhiteboxTools was previously expecting strict adherence to the LAS specifications. I've now re-written the LAS reader to handle extra bytes (ignoring them but not offsetting the other point data) and this seems to have resolved this issue. I've committed the updated code. As such, I am closing this issue. I will be releasing the next version of WhiteboxTools sometime this week, which will have this bug fix. If you happen to test it and encounter any further problems, please feel free to re-open the issue.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jblindsay/whitebox-tools/issues/17?email_source=notifications&email_token=ACYI2WJTJBKWVKXV7J2OPDDPWQYF7A5CNFSM4FTEV7H2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV4VAVY#issuecomment-494489687, or mute the thread https://github.com/notifications/unsubscribe-auth/ACYI2WILNVYDJMSAKNWPIY3PWQYF7ANCNFSM4FTEV7HQ .