Conversion to type double when integer is too big in stdmetrics

[klahssini]

Hello Jean-Romain,

Thank you for the very useful and powerful package lidR. I have been using it for some months now and I am trying to compute standard metrics using the grid_metrics() function, with .stdmetrics as the func argument.

I am having an issue regarding the computation of the total intensity (itot, which is part of stdmetrics_i). The error message is about a column being of type double instead of integer. I copy the error message here for information :

"Error in `[.data.table`(las@data, , if (!anyNA(.BY)) c(eval(call)), by = cells) : 
  Column 1 of result for group 59 is type 'double' but expecting type 'integer'. Column types must be consistent for each group."

I have investigated the issue and it seems that the total intensity, which is defined as the sum of all the returns' intensities, reaches a too big value to be kept as an integer and is seen as a double by R. I tried to "manually" compute the sum of intensities using a custom function rather than .stdmetrics_i in grid_metrics(), and I got the same error. Then I added a conversion of the result as an integer using the as.integer64() function from the bit64 package (which allows conversion of "big" integers), and the issue was solved.

Is there a way to "force" stdmetrics_i to keep the itot result as an integer ? Since I'm computing grid_metrics() over a whole LAScatalog there are only a few chunks where itot is too big, but it ruins the whole computation. Thank you very much for your feedback !

[Jean-Romain]

Interesting problem. I guess there are only two reliable solutions:

• Force itot to be double everywhere every time
• Add a check if itot is greater than integer.max and trigger a warning and assign integer.max as a limit

bit64 is not an option for me because I don't think raster deal with 64 bits integer. Anyway I think a double can hold the numbers.

That being said I'm wondering how you reached such value? Very big pixels? Intensity outiers? Intensities ranging in very high values?

[klahssini]

Thanks for you quick reply ! I am working on a specific LiDAR acquisition and the intensities of returns range in very high values indeed (values are about 1e5 for a single return). I'm unsure of the reason of these high values.

I am computing all standard metrics in a 30m resolution grid, using this call : grid_metrics(lasCatalog, .stdmetrics, res=30). It seems 30m is enough to reach the max value of integer in R, I tried with smaller resolutions (10m and 20m) and I had no issue. So the pixel size along with the very high intensity values would explain it.

I am not sure I understood the first solution you are proposing. Are you suggesting I call stdmetrics_i in my own custom function, where I can force the return to be a double ? Many thanks for your help !

[Jean-Romain]

So the pixel size along with the very high intensity values would explain it.

You are summing all the intensities so yes, larger pixels > more points > higher sum.

I am working on a specific LiDAR acquisition and the intensities of returns range in very high values indeed (values are about 1e5 for a single return)

While this does not solve the problem at his root you can scale the intensities on-the-fly at read time to fix the problem.

readLAS(file, filter = "-scale_intensity 0.01")

For more options see

rlas::read.las(lasfile, transform = "-h")

Maybe -translate_intensity -100000 can do a better job by the way depending on your data.

[klahssini]

Yes this is actually a good fix thanks ! I just need to figure out how to apply the scaling to a LASCatalog object, since I'm not working with LAS object. Thanks for the advice :)

[Jean-Romain]

opt_filter(lascatalog) <- "-scale_intensity 0.01"

[Jean-Romain]

I am not sure I understood the first solution you are proposing. Are you suggesting I call stdmetrics_i in my own custom function, where I can force the return to be a double ? Many thanks for your help !

No, I'm thinking about how to fix the code in lidR.

[klahssini]

Thank you Jean-Romain I was able to scale the intensity using a filter on my LASCatalog. The computation of the intensity metrics worked fine ! Thank you for your availability and help, cheers 👍