search

r-lidar / rlas

R package to read and write las and laz files used to store LiDAR data
https://cran.r-project.org/package=rlas
GNU General Public License v3.0
34 stars 14 forks source link

LASlib error with read.las #43

Closed bi0m3trics closed 3 years ago

bi0m3trics commented 4 years ago

Hi @Jean-Romain Last week a local National Forest received some lidar data acquired by Quantum Spatial in 2018 as part of the US Forest Service-US Geologic Survey (USFS-USGS) partnership (a 3DEP acquisition) and while I can confirm that it opens in everything else I have (e.g., CloudCompare, ENVI, GlobalMapper, LAStools, etc) when I try and open some (not all) of the las files (unaltered, as delivered by the vendor) in windows I get the following error:

R version 3.6.1 (2019-07-05)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252   

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base    

other attached packages:
[1] rlas_1.3.4   lidR_2.1.3   raster_3.0-7 sp_1.3-1   

loaded via a namespace (and not attached):
[1] compiler_3.6.1    lazyeval_0.2.2    tools_3.6.1       Rcpp_1.0.2        codetools_0.2-16  grid_3.6.1      
[7] data.table_1.12.2 lattice_0.20-38 
> las<-rlas::read.las("F:/QL1_UTM12_Deliverables/point_cloud/tilecls/34111H6117.las")
ERROR: reading -989724376 bytes of data into header.evlrs[0].data
ERROR: cannot open lasreaderlas with file name "F:\QL1_UTM12_Deliverables\point_cloud\tilecls\34111H6117.las"
Error in C_reader(ifiles, ofile, select, filter, filter_wkt) :
  LASlib internal error. See message above.

An example file can be downloaded here: https://drive.google.com/open?id=1-gXI9di34qOvz3BQAqSx7EBwQOiZ8c5l

I can also confirm that this problem appears to be windows specific, as the offensive file(s) open fine on my linux machine. I can also confirm that clipping/filtering with las2las and saving also creates a file that can then be read by rlas with no errors, and using the non-commercial versions of lasclip and lasthin read it fine and create non-error files… also lasinfo -i *.las -repair doesn't seem to fix the issue either.

Jean-Romain commented 4 years ago

Could you tell me if read.lasheader() generates the same error.

bi0m3trics commented 4 years ago

It does:

library(rlas) las<-rlas::read.las("F:/QL1_UTM12_Deliverables/point_cloud/tilecls/34111H6117.las") ERROR: reading -989724376 bytes of data into header.evlrs[0].data ERROR: cannot open lasreaderlas with file name 'F:\QL1_UTM12_Deliverables\point_cloud\tilecls\34111H6117.las' Error in C_reader(ifiles, ofile, select, filter, filter_wkt) : LASlib internal error. See message above. read.lasheader("F:/QL1_UTM12_Deliverables/point_cloud/tilecls/34111H6117.las") ERROR: reading -989724376 bytes of data into header.evlrs[0].data ERROR: cannot open lasreaderlas with file name 'F:\QL1_UTM12_Deliverables\point_cloud\tilecls\34111H6117.las' Error: LASlib internal error. See message above. list()

Jean-Romain commented 3 years ago

Good new, after #50 I've updated LASlib and I tried to read your file on Windows. The problem seems fixed. The fix of #50 does not include the update but I guess I will be able to close this issue in few days

bi0m3trics commented 3 years ago

Nice. Saves me the task of opening, saving and closing all files once in a catalog prior to use...

Jean-Romain commented 3 years ago

I pushed an updated version of rlas. Could please test with the current version to confirm that it still fails then test with rlas 1.4.0 and confirm if it works. Hopefully I will be able to close this 1.5 year old issue.

Notice that lidR <= 1.3.1 is not compatible with rlas 1.4.0 so you need lidR 1.3.2 if you want to work with rlas 1.4.0. lidR 1.3.2 is on CRAN but no windows builds are available yet.

bi0m3trics commented 3 years ago

I only have access to my windows machine right now, but it still fails on 1.3.9 and 1.4.0 on the devel branch won't build for me.... If it's helpful, here's my build console output... build_log.txt

Jean-Romain commented 3 years ago

The bug is windows specific. I'm able to read the file with rlas 1.3.9 on linux. This is what I get with 1.3.9 and 1.4.0. We can see that 1.4.0 read the Extended Variable Length Recoards and 1.3.9 doesn't. But this is related to LASlib it comes from improvement in rlas. Contrary to your report the Extended Variable Length Recoards did not prevent me to read the file.

If it's helpful, here's my build console output...

It compiles in C++11 despite the Makefle.win states it should compile in C++14. Did you modiy your .R/Makevars file (or windows equivalent file) manually?

rlas::read.lasheader("~/Téléchargements/issue 43/34111H6117.las")
#> $`File Signature`
#> [1] "LASF"
#> $`File Source ID`
#> [1] 0
#> $`Global Encoding`
#> $`Global Encoding`$`GPS Time Type`
#> [1] TRUE
#> $`Global Encoding`$`Waveform Data Packets Internal`
#> [1] FALSE
#> $`Global Encoding`$`Waveform Data Packets External`
#> [1] FALSE
#> $`Global Encoding`$`Synthetic Return Numbers`
#> [1] FALSE
#> $`Global Encoding`$WKT
#> [1] TRUE
#> $`Global Encoding`$`Aggregate Model`
#> [1] FALSE
#> $`Project ID - GUID`
#> [1] "00000000-0000-0000-0000-000000000000"
#> $`Version Major`
#> [1] 1
#> $`Version Minor`
#> [1] 4
#> $`System Identifier`
#> [1] "Quantum Spatial"
#> $`Generating Software`
#> [1] "LiDAR Suite"
#> $`File Creation Day of Year`
#> [1] 241
#> $`File Creation Year`
#> [1] 2019
#> $`Header Size`
#> [1] 375
#> $`Offset to point data`
#> [1] 519
#> $`Number of variable length records`
#> [1] 2
#> $`Point Data Format ID`
#> [1] 6
#> $`Point Data Record Length`
#> [1] 30
#> $`Number of point records`
#> [1] 71656308
#> $`Number of points by return`
#>  [1] 38258506 20321461  9495283  2905490   585391    81327     8188      617
#>  [9]       45        0        0        0        0        0        0
#> $`X scale factor`
#> [1] 0.01
#> $`Y scale factor`
#> [1] 0.01
#> $`Z scale factor`
#> [1] 0.01
#> $`X offset`
#> [1] 0
#> $`Y offset`
#> [1] 0
#> $`Z offset`
#> [1] 0
#> $`Max X`
#> [1] 433811.7
#> $`Min X`
#> [1] 432660.4
#> $`Max Y`
#> [1] 3869133
#> $`Min Y`
#> [1] 3867738
#> $`Max Z`
#> [1] 2155.38
#> $`Min Z`
#> [1] 1796.07
#> $`Variable Length Records`
#> $`Variable Length Records`$NIIRS10
#> $`Variable Length Records`$NIIRS10$reserved
#> [1] 0
#> $`Variable Length Records`$NIIRS10$`user ID`
#> [1] "NIIRS10"
#> $`Variable Length Records`$NIIRS10$`record ID`
#> [1] 4
#> $`Variable Length Records`$NIIRS10$`length after header`
#> [1] 10
#> $`Variable Length Records`$NIIRS10$description
#> [1] "NIIRS10 Timestamp"
#> $`Variable Length Records`$NIIRS10
#> $`Variable Length Records`$NIIRS10$reserved
#> [1] 0 
#> $`Variable Length Records`$NIIRS10$`user ID`
#> [1] "NIIRS10"
#> $`Variable Length Records`$NIIRS10$`record ID`
#> [1] 1
#> $`Variable Length Records`$NIIRS10$`length after header`
#> [1] 26
#> $`Variable Length Records`$NIIRS10$description
#> [1] "NIIRS10 Tile Index"
rlas::read.lasheader("~/Téléchargements/issue 43/34111H6117.las")
#> $`File Signature`
#> [1] "LASF"
#> $`File Source ID`
#> [1] 0
#> $`Global Encoding`
#> $`Global Encoding`$`GPS Time Type`
#> [1] TRUE
#> $`Global Encoding`$`Waveform Data Packets Internal`
#> [1] FALSE
#> $`Global Encoding`$`Waveform Data Packets External`
#> [1] FALSE
#> $`Global Encoding`$`Synthetic Return Numbers`
#> [1] FALSE
#> $`Global Encoding`$WKT
#> [1] TRUE
#> $`Global Encoding`$`Aggregate Model`
#> [1] FALSE
#> $`Project ID - GUID`
#> [1] "00000000-0000-0000-0000-000000000000"
#> $`Version Major`
#> [1] 1
#> $`Version Minor`
#> [1] 4
#> $`System Identifier`
#> [1] "Quantum Spatial"
#> $`Generating Software`
#> [1] "LiDAR Suite"
#> $`File Creation Day of Year`
#> [1] 241
#> $`File Creation Year`
#> [1] 2019
#> $`Header Size`
#> [1] 375
#> $`Offset to point data`
#> [1] 519
#> $`Number of variable length records`
#> [1] 2
#> $`Point Data Format ID`
#> [1] 6
#> $`Point Data Record Length`
#> [1] 30
#> $`Number of point records`
#> [1] 71656308
#> $`Number of points by return`
#>  [1] 38258506 20321461  9495283  2905490   585391    81327     8188      617
#>  [9]       45        0        0        0        0        0        0
#> $`X scale factor`
#> [1] 0.01
#> $`Y scale factor`
#> [1] 0.01
#> $`Z scale factor`
#> [1] 0.01
#> $`X offset`
#> [1] 0
#> $`Y offset`
#> [1] 0
#> $`Z offset`
#> [1] 0
#> $`Max X`
#> [1] 433811.7
#> $`Min X`
#> [1] 432660.4
#> $`Max Y`
#> [1] 3869133
#> $`Min Y`
#> [1] 3867738
#> $`Max Z`
#> [1] 2155.38
#> $`Min Z`
#> [1] 1796.07
#> $`Variable Length Records`
#> $`Variable Length Records`$NIIRS10
#> $`Variable Length Records`$NIIRS10$reserved
#> [1] 0
#> $`Variable Length Records`$NIIRS10$`user ID`
#> [1] "NIIRS10"
#> $`Variable Length Records`$NIIRS10$`record ID`
#> [1] 4
#> $`Variable Length Records`$NIIRS10$`length after header`
#> [1] 10
#> $`Variable Length Records`$NIIRS10$description
#> [1] "NIIRS10 Timestamp"
#> $`Variable Length Records`$NIIRS10
#> $`Variable Length Records`$NIIRS10$reserved
#> [1] 0
#> $`Variable Length Records`$NIIRS10$`user ID`
#> [1] "NIIRS10"
#> $`Variable Length Records`$NIIRS10$`record ID`
#> [1] 1
#> $`Variable Length Records`$NIIRS10$`length after header`
#> [1] 26
#> $`Variable Length Records`$NIIRS10$description
#> [1] "NIIRS10 Tile Index"
#> $`Extended Variable Length Records`
#> $`Extended Variable Length Records`$`WKT OGC CS`
#> $`Extended Variable Length Records`$`WKT OGC CS`$reserved
#> [1] 0
#> $`Extended Variable Length Records`$`WKT OGC CS`$`user ID`
#> [1] "LASF_Projection"
#> $`Extended Variable Length Records`$`WKT OGC CS`$`record ID`
#> [1] 2112
#> $`Extended Variable Length Records`$`WKT OGC CS`$`length after header`
#> [1] 915
#> $`Extended Variable Length Records`$`WKT OGC CS`$description
#> [1] "WKT Projection"
#> $`Extended Variable Length Records`$`WKT OGC CS`$`WKT OGC COORDINATE SYSTEM`
#> [1] "COMPD_CS[\"NAD83(2011) / UTM zone 12N + NAVD88 height - Geoid12B (meters)\",PROJCS[\"NAD83(2011) / UTM zone 12N\",GEOGCS[\"NAD83(2011)\",DATUM[\"NAD83 (National Spatial Reference System 2011)\",SPHEROID[\"GRS 1980\",6378137,298.257222101,AUTHORITY[\"EPSG\",\"7019\"]],AUTHORITY[\"EPSG\",\"1116\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"6318\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-111],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],UNIT[\"meter\",1,AUTHORITY[\"EPSG\",\"9001\"]],AXIS[\"X\",EAST],AXIS[\"Y\",NORTH],AUTHORITY[\"EPSG\",\"6341\"]],VERT_CS[\"NAVD88 height - Geoid12B (meters)\",VERT_DATUM[\"North American Vertical Datum 1988\",2005,AUTHORITY[\"EPSG\",\"5103\"]],UNIT[\"meter\",1,AUTHORITY[\"EPSG\",\"9001\"]],AXIS[\"Up\",UP],AUTHORITY[\"EPSG\",\"5703\"]]]"

Created on 2021-03-21 by the reprex package (v1.0.0)

bi0m3trics commented 3 years ago

What the hell!??!?! I get up this morning and remotes::install_github("Jean-Romain/rlas") builds on windows fine for me now... and it looks like nothing changed! Maybe I need to go back to sleep...

Anyway, here's what I can tell you.... > sessionInfo()

R version 4.0.4 (2021-02-15)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 16299)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    
attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     
other attached packages:
[1] rlas_1.4.0
loaded via a namespace (and not attached):
 [1] Rcpp_1.0.6        ps_1.6.0          prettyunits_1.1.1 crayon_1.4.1      withr_2.4.1      
 [6] rprojroot_2.0.2   assertthat_0.2.1  R6_2.5.0          cli_2.3.1         curl_4.3         
[11] remotes_2.2.0     rstudioapi_0.13   callr_3.5.1       tools_4.0.4       glue_1.4.2       
[16] compiler_4.0.4    processx_3.4.5    pkgbuild_1.2.0

> rlas::read.lasheader("C:/Users/ajm69/Downloads/34111H6117.las")

$`File Signature`
[1] "LASF"
$`File Source ID`
[1] 0
$`Global Encoding`
$`Global Encoding`$`GPS Time Type`
[1] TRUE
$`Global Encoding`$`Waveform Data Packets Internal`
[1] FALSE
$`Global Encoding`$`Waveform Data Packets External`
[1] FALSE
$`Global Encoding`$`Synthetic Return Numbers`
[1] FALSE
$`Global Encoding`$WKT
[1] TRUE
$`Global Encoding`$`Aggregate Model`
[1] FALSE
$`Project ID - GUID`
[1] "00000000-0000-0000-0000-000000000000"
$`Version Major`
[1] 1
$`Version Minor`
[1] 4
$`System Identifier`
[1] "Quantum Spatial"
$`Generating Software`
[1] "LiDAR Suite"
$`File Creation Day of Year`
[1] 241
$`File Creation Year`
[1] 2019
$`Header Size`
[1] 375
$`Offset to point data`
[1] 519
$`Number of variable length records`
[1] 2
$`Point Data Format ID`
[1] 6
$`Point Data Record Length`
[1] 30
$`Number of point records`
[1] 71656308
$`Number of points by return`
 [1] 38258506 20321461  9495283  2905490   585391    81327     8188      617       45        0        0
[12]        0        0        0        0
$`X scale factor`
[1] 0.01
$`Y scale factor`
[1] 0.01
$`Z scale factor`
[1] 0.01
$`X offset`
[1] 0
$`Y offset`
[1] 0
$`Z offset`
[1] 0
$`Max X`
[1] 433811.7
$`Min X`
[1] 432660.4
$`Max Y`
[1] 3869133
$`Min Y`
[1] 3867738
$`Max Z`
[1] 2155.38
$`Min Z`
[1] 1796.07
$`Variable Length Records`
$`Variable Length Records`$NIIRS10
$`Variable Length Records`$NIIRS10$reserved
[1] 0
$`Variable Length Records`$NIIRS10$`user ID`
[1] "NIIRS10"
$`Variable Length Records`$NIIRS10$`record ID`
[1] 4
$`Variable Length Records`$NIIRS10$`length after header`
[1] 10
$`Variable Length Records`$NIIRS10$description
[1] "NIIRS10 Timestamp"
$`Variable Length Records`$NIIRS10
$`Variable Length Records`$NIIRS10$reserved
[1] 0
$`Variable Length Records`$NIIRS10$`user ID`
[1] "NIIRS10"
$`Variable Length Records`$NIIRS10$`record ID`
[1] 1
$`Variable Length Records`$NIIRS10$`length after header`
[1] 26
$`Variable Length Records`$NIIRS10$description
[1] "NIIRS10 Tile Index"
$`Extended Variable Length Records`
$`Extended Variable Length Records`$`WKT OGC CS`
$`Extended Variable Length Records`$`WKT OGC CS`$reserved
[1] 0
$`Extended Variable Length Records`$`WKT OGC CS`$`user ID`
[1] "LASF_Projection"
$`Extended Variable Length Records`$`WKT OGC CS`$`record ID`
[1] 2112
$`Extended Variable Length Records`$`WKT OGC CS`$`length after header`
[1] 915
$`Extended Variable Length Records`$`WKT OGC CS`$description
[1] "WKT Projection"
$`Extended Variable Length Records`$`WKT OGC CS`$`WKT OGC COORDINATE SYSTEM`
[1] "COMPD_CS[\"NAD83(2011) / UTM zone 12N + NAVD88 height - Geoid12B (meters)\",PROJCS[\"NAD83(2011) / UTM zone 12N\",GEOGCS[\"NAD83(2011)\",DATUM[\"NAD83 (National Spatial Reference System 2011)\",SPHEROID[\"GRS 1980\",6378137,298.257222101,AUTHORITY[\"EPSG\",\"7019\"]],AUTHORITY[\"EPSG\",\"1116\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"6318\"]],PROJECTION[\"Transverse_Mercator\"],PARAMETER[\"latitude_of_origin\",0],PARAMETER[\"central_meridian\",-111],PARAMETER[\"scale_factor\",0.9996],PARAMETER[\"false_easting\",500000],PARAMETER[\"false_northing\",0],UNIT[\"meter\",1,AUTHORITY[\"EPSG\",\"9001\"]],AXIS[\"X\",EAST],AXIS[\"Y\",NORTH],AUTHORITY[\"EPSG\",\"6341\"]],VERT_CS[\"NAVD88 height - Geoid12B (meters)\",VERT_DATUM[\"North American Vertical Datum 1988\",2005,AUTHORITY[\"EPSG\",\"5103\"]],UNIT[\"meter\",1,AUTHORITY[\"EPSG\",\"9001\"]],AXIS[\"Up\",UP],AUTHORITY[\"EPSG\",\"5703\"]]]"

and then with 3.2.0 (this tile has a ton of noise)... > lidR::plot(las, backend="lidRviewer") Capture

So... it looks like you can close this one to me! Thanks for keeping at it!

THLAB35 commented 2 years ago

Hello,

I encounter this error:

LASlib internal error. See message above. In addition: Warning message: lasnormalize is deprecated. Use normalize_height instead.

Can you help me?

Thank you,

Jean-Romain commented 2 years ago

It does not work is not enough to help. Please open an issue