Closed serbinsh closed 6 years ago
@akamoske never mind, perhaps it is working now? not sure why i originally had another error? OK yes it created the output!
So before we merge this (if we do) lets make sure we clean up the commenting. Just there temporarily until you think what I changed makes sense
Hmm...for whatever reason when I run the function sometimes I get this
> # Convert .laz or .las files into a list of voxelized lidar arrays
> laz.data <- laz.to.array(laz.files.path = "Downloads",
+ voxel.resolution = 10, z.resolution = 1)
Error in laz.xyz[, 1] : incorrect number of dimensions
>
but I can go line by line just fine?
Ugh! But if I open the function .R file and "source" it then run that function again it works fine?!?!??!! But I am R CMD build / R CMD INSTALL each time so it should be getting the newer function file. I have not idea why it wont work when called unless I source it first.
Ooops likes like another NAMESPACE issue:
lad.estimates <- MacHorn.LAD(level.canopy, 1, NULL) Error in MacHorn.LAD(level.canopy, 1, NULL) : could not find function "MacHorn.LAD"
@akamoske other errors I am seeing
> lad.estimates <- MacHorn.LAD(level.canopy, 1, NULL)
Error in MacHorn.LAD(level.canopy, 1, NULL) :
could not find function "MacHorn.LAD"
> # Estimate LAD for each voxel in leveled array in the list
> lad.estimates <- machorn.lad(level.canopy, 1, NULL)
> # Convert the list of LAD arrays into a single raster stack
> lad.raster <- lad.array.to.raster.stack(lad.estimates, 32618)
[1] "Raster number 1 is complete!"
Error in as.data.frame.default(x) :
cannot coerce class ‘structure("RasterStack", package = "raster")’ to a data.frame
I fixed the first by changing to machorn.lad()
Shawn, I get the same problem with
Convert .laz or .las files into a list of voxelized lidar arrays
laz.data <- laz.to.array(laz.files.path = "Downloads",
- voxel.resolution = 10, z.resolution = 1) Error in laz.xyz[, 1] : incorrect number of dimensions
when I run this! This is what is driving me insane. If I run it line by line it works fine, but as a function is gets weird...
Good eye on the Machorn.LAD though, going to change that right now. I'm at a lose on the incorrect numbers of dimensions though.
@akamoske for some reason when called in the function laz.xyz comes up with:
[1] "Processing: Downloads/NEON_D01_HARV_DP1_731000_4712000_classified_point_cloud.laz"
Error in laz.xyz[, 1] : incorrect number of dimensions
Enter a frame number, or 0 to exit
1: laz.to.array(laz.files.path = "Downloads", voxel.resolution = 10, z.resolution = 1)
Selection: 1
Called from: top level
Browse[1]> ls()
[1] "i" "laz.data" "laz.file" "laz.files.list" "laz.files.path" "laz.list" "laz.xyz"
[8] "laz.xyz.table" "voxel.resolution" "z.resolution"
Browse[1]> i
[1] 1
Browse[1]> laz.xyz
numeric(0)
trying to debug to see why that is...but this is a pretty difficult error and very hard to track down
Interesting, if I look at xyz.table I see this
rowse[1]> head(laz.xyz.table)
c(laz.data[, "X"], laz.data[, "Y"], laz.data[, "Z"], laz.data[, "Classification"])
1 731999.9
2 731999.9
3 731999.9
4 731999.8
5 731999.8
6 731999.8
which is clearly wrong but if run line by line it is correctly produced so looking into why
@akamoske I may have solved the incorrect dimensions error:
> # Convert .laz or .las files into a list of voxelized lidar arrays
> laz.data <- laz.to.array(laz.files.path = "Downloads",
+ voxel.resolution = 10, z.resolution = 1)
[1] "Processing: Downloads/NEON_D01_HARV_DP1_731000_4712000_classified_point_cloud.laz"
|===============================================================================================================================| 100%
> head(laz.data)
[[1]]
[[1]]$array
, , 1
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14] [,15] [,16] [,17] [,18]
[1,] 333.20 332.14 331.79 330.75 330.76 331.04 332.24 331.00 330.87 330.69 329.96 329.16 328.95 328.34 329.14 329.67 332.54 325.58
[2,] 359.47 360.70 360.70 360.99 359.57 353.16 353.53 351.57 353.72 353.75 349.91 351.14 350.01 354.67 358.51 358.33 357.08 355.72
[3,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[4,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[5,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[6,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 19.00
[7,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 71.00
[8,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 38.00
[9,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.00 11.00 0.00 0.00 0.00 20.00
[10,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.00 51.00 65.00 141.00 13.00 5.00 0.00 25.00
[,19] [,20] [,21] [,22] [,23] [,24] [,25] [,26] [,27] [,28] [,29] [,30] [,31] [,32] [,33] [,34] [,35] [,36]
[1,] 325.34 326.21 326.71 327.06 328.54 328.23 328.32 328.39 326.32 325.80 325.55 325.07 324.60 325.11 325.26 324.92 324.71 323.80
[2,] 349.24 345.23 345.61 346.02 341.85 333.17 335.36 348.59 348.77 348.24 346.44 349.30 351.16 347.98 346.22 339.21 348.45 350.11
[3,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[4,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 9.00
[5,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.00 0.00 0.00 2.00 5.00 85.00
[6,] 64.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 10.00 137.00 140.00 147.00 268.00 135.00 75.00 5.00 5.00
[7,] 113.00 181.00 15.00 0.00 0.00 0.00 0.00 0.00 4.00 144.00 65.00 16.00 12.00 21.00 14.00 35.00 4.00 6.00
[8,] 27.00 95.00 105.00 5.00 0.00 0.00 0.00 0.00 6.00 6.00 1.00 0.00 4.00 0.00 1.00 49.00 1.00 0.00
[9,] 2.00 8.00 19.00 11.00 61.00 84.00 149.00 54.00 18.00 5.00 2.00 0.00 0.00 0.00 0.00 23.00 2.00 0.00
[10,] 9.00 0.00 3.00 49.00 166.00 212.00 154.00 10.00 7.00 1.00 11.00 2.00 1.00 8.00 1.00 22.00 7.00 0.00
[,37] [,38] [,39] [,40] [,41] [,42] [,43] [,44] [,45] [,46] [,47] [,48] [,49] [,50] [,51] [,52] [,53] [,54]
[1,] 323.73 323.26 323.46 323.82 324.39 NA 325.58 326.41 327.44 327.81 328.37 327.89 328.31 329.31 330.30 330.74 330.98 331.13
[2,] 346.25 352.98 352.92 356.60 358.23 356.5 357.85 359.57 358.87 355.56 355.39 356.23 360.60 360.89 362.85 361.15 364.12 360.48
[3,] 0.00 0.00 0.00 0.00 0.00 0.0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[4,] 15.00 17.00 4.00 1.00 0.00 0.0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[5,] 113.00 46.00 11.00 6.00 2.00 0.0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[6,] 28.00 4.00 1.00 21.00 6.00 1.0 2.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[7,] 4.00 1.00 0.00 0.00 0.00 0.0 17.00 14.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[8,] 16.00 0.00 0.00 0.00 0.00 0.0 0.00 5.00 5.00 1.00 0.00 1.00 0.00 0.00 0.00 0.00 0.00 0.00
[9,] 23.00 36.00 0.00 0.00 0.00 0.0 0.00 0.00 32.00 26.00 9.00 7.00 16.00 0.00 0.00 0.00 0.00 0.00
[10,] 6.00 6.00 0.00 0.00 0.00 0.0 0.00 0.00 1.00 1.00 0.00 0.00 11.00 23.00 0.00 0.00 0.00 0.00
[,55] [,56] [,57] [,58] [,59] [,60] [,61] [,62] [,63] [,64] [,65] [,66] [,67] [,68] [,69] [,70] [,71] [,72]
[1,] 331.09 331.90 332.64 333.36 332.70 332.88 333.26 333.15 333.49 333.92 334.61 334.66 336.39 335.84 337.28 NA 338.37 340.05
[2,] 359.31 361.71 363.92 365.06 364.48 363.28 362.62 365.80 362.74 362.69 363.57 363.50 363.91 362.74 364.21 364.13 363.38 361.37
[3,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[4,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[5,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[6,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[7,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[8,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[9,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[10,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[,73] [,74] [,75] [,76] [,77] [,78] [,79] [,80] [,81] [,82] [,83] [,84] [,85] [,86] [,87] [,88] [,89] [,90]
[1,] 339.75 340.03 340.38 340.57 340.95 340.62 341.21 341.55 340.96 342.08 NA 342.87 342.89 343.75 344.54 345.50 NA 345.73
[2,] 359.94 359.56 359.45 357.83 357.62 362.97 367.82 367.38 372.31 371.29 371.17 371.21 371.73 364.97 371.32 376.07 374.42 373.52
[3,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[4,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[5,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[6,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[7,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[8,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[9,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[10,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[,91] [,92] [,93] [,94] [,95] [,96] [,97] [,98] [,99] [,100]
[1,] 344.97 344.09 343.23 343.52 343.16 343.82 345.06 345.80 347.66 348.45
[2,] 376.25 374.26 370.55 370.04 368.58 375.65 378.55 372.97 372.41 373.73
[3,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[4,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[5,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[6,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[7,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[8,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[9,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[10,] 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
[ reached getOption("max.print") -- omitted 78 row(s) and 99 matrix slice(s) ]
[[1]]$x.bin
[1] 731000 731010 731020 731030 731040 731050 731060 731070 731080 731090 731100 731110 731120 731130 731140 731150 731160 731170
[19] 731180 731190 731200 731210 731220 731230 731240 731250 731260 731270 731280 731290 731300 731310 731320 731330 731340 731350
[37] 731360 731370 731380 731390 731400 731410 731420 731430 731440 731450 731460 731470 731480 731490 731500 731510 731520 731530
[55] 731540 731550 731560 731570 731580 731590 731600 731610 731620 731630 731640 731650 731660 731670 731680 731690 731700 731710
[73] 731720 731730 731740 731750 731760 731770 731780 731790 731800 731810 731820 731830 731840 731850 731860 731870 731880 731890
[91] 731900 731910 731920 731930 731940 731950 731960 731970 731980 731990 732000
[[1]]$y.bin
[1] 4712000 4712010 4712020 4712030 4712040 4712050 4712060 4712070 4712080 4712090 4712100 4712110 4712120 4712130 4712140 4712150
[17] 4712160 4712170 4712180 4712190 4712200 4712210 4712220 4712230 4712240 4712250 4712260 4712270 4712280 4712290 4712300 4712310
[33] 4712320 4712330 4712340 4712350 4712360 4712370 4712380 4712390 4712400 4712410 4712420 4712430 4712440 4712450 4712460 4712470
[49] 4712480 4712490 4712500 4712510 4712520 4712530 4712540 4712550 4712560 4712570 4712580 4712590 4712600 4712610 4712620 4712630
[65] 4712640 4712650 4712660 4712670 4712680 4712690 4712700 4712710 4712720 4712730 4712740 4712750 4712760 4712770 4712780 4712790
[81] 4712800 4712810 4712820 4712830 4712840 4712850 4712860 4712870 4712880 4712890 4712900 4712910 4712920 4712930 4712940 4712950
[97] 4712960 4712970 4712980 4712990 4713000
[[1]]$z.bin
[1] 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355
[34] 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388
[67] 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408
Ok now looking at this issue
> lad.raster <- lad.array.to.raster.stack(lad.estimates, 32618)
[1] "Raster number 1 is complete!"
Error in as.data.frame.default(x) :
cannot coerce class ‘structure("RasterStack", package = "raster")’ to a data.frame
@akamoske I am trying to understand some of the logic in the code and it seems that:
lai.raster <- raster::calc(lad.raster, fun = sum, na.rm = TRUE)
doesnt make sense when using only 1 laz file? Do you normally run this code with a whole bunch of laz files in a dir? I had assumed that .laz files had all canopy layers but then not sure what all the "stacks" are that are being generated.
That calc function throws an error if you only have 1 input laz file. I figured this out by faking more than one list by doing
lad.rstack.list[[1]] <- lad.rasters
lad.rstack.list[[2]] <- lad.rasters
then
lad.ras <- base::do.call(raster::merge, lad.rstack.list)
lad.raster <- lad.ras
lai.raster <- raster::calc(lad.raster, fun = sum, na.rm = TRUE)
which then works. So would you normally have a laz for each slice of the canopy? That is each layer?
@akamoske here is my final commit to this PR. Check it out to consider merging with master. Perhaps check out this branch and test yourself to be sure nothing is broken. With these changes the code now works on my end.
Could you share another .laz file so i can be sure it is stacking properly? I just made another copy of the file you send and renamed it so that the full set of example function calls would run.
FYI - I tested this branch as follows:
first I installed my branch: install_github("serbinsh/LiDARforestR", ref="serbinsh.branch")
> library(devtools)
> install_github("serbinsh/LiDARforestR", ref="serbinsh.branch")
Downloading GitHub repo serbinsh/LiDARforestR@serbinsh.branch
from URL https://api.github.com/repos/serbinsh/LiDARforestR/zipball/serbinsh.branch
Installing LiDARforestR
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ --no-save --no-restore --quiet CMD INSTALL \
'/private/var/folders/9g/r6pn2vsj2m3gyc6bqcz5s7_80000gn/T/Rtmp0f1QEO/devtools2abb37a105ca/serbinsh-LiDARforestR-d7a1a8c' \
--library='/Users/shawnserbin/R/library' --install-tests
* installing *source* package ‘LiDARforestR’ ...
** R
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (LiDARforestR)
>
library(LiDARforestR)
laz.data <- laz.to.array(laz.files.path = "Downloads",
voxel.resolution = 10, z.resolution = 1)
level.canopy <- canopy.height.levelr(laz.data)
lad.estimates <- machorn.lad(level.canopy, 1, NULL)
lad.raster <- lad.array.to.raster.stack(lad.estimates, 32618)
lai.raster <- raster::calc(lad.raster, fun = sum, na.rm = TRUE)
plot(lai.raster)
Shawn you are a wizard!!!
Just ran through it all and it works great! Its so weird that it was running fine through the source, but once it was loaded as a package it was throwing these errors...
I'm going to merge this with the master and do a bit of clean up.
Thanks so much!!!!!!!
Hey Shawn, I just merged it all together. Also, in response to your question about the LAI raster creation from the LAD raster stack. It shouldn't matter if there is only one laz file or not. The raster stack is a stack of rasters from the LAD array, where each raster in the stack is a 1 meter slice of the canopy. So it takes the sum of the same pixel for each raster in the stack, thus adding together all the LAD values within a vertical column of voxels. Hopefully that kinda makes sense...
Hey Shawn. I just added a filter to keep .laz or .las files that are less than 100kb from going through the processing chain. Basically NEON has a couple of empty files in there that cause the code to crash and this removes them. I tested in on Windows and can test it on my Ubuntu machine later today. I was wondering if you could just do a quick test on your Apple to make sure that it works ok?
I just added this line of code below the list.files command:
laz.files.list <- list.files(laz.files.path, pattern=c("\.laz$|.las$"), full.names = TRUE)
laz.files.list <- laz.files.list[sapply(laz.files.list, file.size) > 100000]
Thanks!!!
@akamoske checking now by running your example in the README, is that sufficient for testing on mac osx?
Should be! Just wanted to make sure there wasn't some type of weird error that would pop up.
I'm running through the 2017 SERC data right now, there are 130 .laz files (each one is 1x1 km), so this will be a good test with a very large list of .laz files
Strange, I am getting this on my mac at work?
> scratch_folder <- file.path("~/scratch/neon_data/")
> if (! file.exists(scratch_folder)) dir.create(scratch_folder,recursive=TRUE)
> setwd(file.path(scratch_folder))
> getwd()
[1] "/Users/sserbin/scratch/neon_data"
>
>
> laz.data <- laz.to.array(laz.files.path = "./",
+ voxel.resolution = 10,
+ z.resolution = 1)
[1] "Processing: .//neon_lidar_example.las"
Error: 'readlasdata' is not an exported object from 'namespace:rlas'
>
what version of rlas are you using?
@akamoske looking here (https://github.com/Jean-Romain/rlas/blob/master/NAMESPACE) I dont see readlasdata in the NAMESPACE of rlas. Perhaps this is a new change in rlas package?
I'm using rlas_1.2.4
It looks like they just updated to 1.2.7 on 8/20/18 and condensed some of the functions down. There was a period where they had a couple different options.
Hopefully this will be stable now, I'll update the code and push it real quick.
Yeah, looks diff now:
> rlas::read.las
rlas::read.las rlas::read.lasheader
> rlas::read.las
They are supposed to provide back compatibility and use warnings that certain calls are deprecated before just removing functions or changing names. That is annoying and bad R coding
Just pushed it I'll do a quick test over here
Super annoying.
Works fine on my end!
OK trying again
Cool. Also, I just ran through the entire 2017 NEON LiDAR dataset for SERC (130 tiles = 130 km2). Took about 1.5 hours to run through all the code, while saving RData and .tif files along the way.
OK great, working for me too!
cool!
Thanks again!
@akamoske I made this change....let me know what you think? This solved the issue of not finding the files but not sure this is what you intended. I ran into another error when running the function. trying to sort out where.