search

r-lidar / lasR

Fast and Pipable Airborne LiDAR Data Tools
https://r-lidar.github.io/lasR/
GNU General Public License v3.0
63 stars 1 forks source link

malloc() Arrow and Parquet Error #20

Closed kalinowskapatrycja closed 7 months ago

kalinowskapatrycja commented 7 months ago

I get the following error:

$ Rscript process.r ST3_A01_2018_P04_LAS_2.laz
ERROR 1: /usr/lib/gdalplugins/ogr_Arrow.so: undefined symbol: _ZNK5arrow16KeyValueMetadata3GetERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
ERROR 1: /usr/lib/gdalplugins/ogr_Arrow.so: undefined symbol: _ZNK5arrow16KeyValueMetadata3GetERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
ERROR 1: /usr/lib/gdalplugins/ogr_Parquet.so: undefined symbol: _ZNK5arrow16KeyValueMetadata8ContainsERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
ERROR 1: /usr/lib/gdalplugins/ogr_Parquet.so: undefined symbol: _ZNK5arrow16KeyValueMetadata8ContainsERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
malloc(): invalid next size (unsorted)
zsh: IOT instruction (core dumped)  Rscript process.r ST3_A01_2018_P04_LAS_2.laz

When running this code on the tutorial page:

ibrary(lasR)

args = commandArgs(trailingOnly = TRUE)
f = args[1]

del = triangulate(filter = keep_first())
chm = rasterize(0.5, del)
chm2 = pit_fill(chm)
seed = local_maximum_raster(chm2, 3)
tree = region_growing(chm2, seed)
pipeline = del + chm + chm2 +  seed + tree
ans = exec(pipeline, on = f)

col = grDevices::colorRampPalette(c("blue", "cyan2", "yellow", "red"))(25)
col2 = grDevices::colorRampPalette(c("purple", "blue", "cyan2", "yellow", "red", "green"))(50)
terra::plot(ans$rasterize, col = col, mar = c(1, 1, 1, 3))
terra::plot(ans$pit_fill, col = col, mar = c(1, 1, 1, 3))
terra::plot(ans$region_growing, col = col2[sample.int(50, 277, TRUE)], mar = c(1, 1, 1, 3))
plot(ans$local_maximum$geom, add = T, pch = 19, cex = 0.5)

I am using this file: LAZ.Example.zip

The previous example on the tutorial page works fine. gdal and arrow are installed.

Jean-Romain commented 7 months ago

I confirm the issue. Thank you for reporting.

f = "/home/jr/Téléchargements/issue 20/ST3_A01_2018_P04_LAS_2.laz"

chm = rasterize(0.5, "max")
seed = local_maximum_raster(chm, 3)
pipeline = chm +  seed
ans = exec(pipeline, on = f, noread = T)
==371706== Invalid write of size 8
==371706==    at 0xCC282DB: memcpy (string_fortified.h:29)
==371706==    by 0xCC282DB: copy_to (laspoint.hpp:231)
==371706==    by 0xCC282DB: LAS::add_point(LASpoint const&) (LAS.cpp:153)
==371706==    by 0xCC2A5DE: LAS::LAS(Raster const&) (LAS.cpp:102)
==371706==    by 0xCC6074D: LASRlocalmaximum::process() (localmaximum.cpp:62)
==371706==    by 0xCC551F6: Pipeline::run_loaded() (pipeline.cpp:180)
==371706==    by 0xCC56484: Pipeline::run() (pipeline.cpp:92)
==371706==    by 0xCD4102C: process(SEXPREC*, SEXPREC*) [clone ._omp_fn.0] (process.cpp:197)
==371706==    by 0x572CA15: GOMP_parallel (in /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0)
==371706==    by 0xCD416FB: process(SEXPREC*, SEXPREC*) (process.cpp:142)
==371706==    by 0x49582C9: ??? (in /usr/lib/R/lib/libR.so)
==371706==    by 0x499B272: ??? (in /usr/lib/R/lib/libR.so)
==371706==    by 0x49AFCFF: Rf_eval (in /usr/lib/R/lib/libR.so)
==371706==    by 0x49B1D85: ??? (in /usr/lib/R/lib/libR.so)
==371706==  Address 0x2064fe10 is not stack'd, malloc'd or (recently) free'd
==371706== 

 *** caught segfault ***
address 0x2064fe10, cause 'memory not mapped'
Jean-Romain commented 7 months ago

Very stupid bug. I should have tested the code on a bigger example. Fixed now. Thanks

kalinowskapatrycja commented 7 months ago

Thank you!

May I ask whether the error attached is an issue with lasR or my system? I have reinstalled lasR, gdal and arrow with no avail.

errors.txt

Jean-Romain commented 7 months ago

Not related to me. At least not directly. I don't know what is the problem. The package is supposed to compile on windows according to the continuous integration with github action https://github.com/r-lidar/lasR/actions/runs/8511576203

What is weird here is that it compiled for you previously. Nothing changed since lasR 0.1.0 with respect to gdal.

Jean-Romain commented 7 months ago

Well, according to your paths you are on linux. I can try to help but have no idea of the problem. You were able to install 0.4.0 and nothing change except 2 lines of code to fix two simple bugs

kalinowskapatrycja commented 7 months ago

Well, according to your paths you are on linux. I can try to help but have no idea of the problem. You were able to install 0.4.0 and nothing change except 2 lines of code to fix two simple bugs

Well it looks like everything is functioning correctly even with the error. So I may just ignore this for now. It does look like an issue with my system.

Jean-Romain commented 7 months ago

Mmm it was not an issue at the end of the compilation while linking?

kalinowskapatrycja commented 7 months ago

Mmm it was not an issue at the end of the compilation while linking?

I am not too sure what that means unfortunately. How do I find out

I only had this warning when installing lasR image

PS, does lasR support the feature in lidR - 7.1.2 Local Maximum Filter with variable windows size? I am struggling to get good tree detection without the use of this function (Smaller trees are given huge canopy sizes so my entire plot is covered in canopy even if the .laz has visible ground) image image

Also this link no longer seems to be working in the Parallel Processing section https://r-lidar.github.io/lasR/benchmarks.html Though I don't strictly need it, just wanted to let you know.

Jean-Romain commented 7 months ago

PS, does lasR support the feature in lidR - 7.1.2 Local Maximum Filter with variable windows size?

No it does not. One of the main reasons is because it is much slower to compute. The second reason is that I must register a set of pre-defined functions in c++. Out of question to allow a user-defined function like in lidR it is too complex. Overall I'm keen to do it but it is not my priority.

Also this link no longer seems to be working in the Parallel Processing section https://r-lidar.github.io/lasR/benchmarks.html Though I don't strictly need it, just wanted to let you know.

This link in incorrect. Where did you find it? Thanks

kalinowskapatrycja commented 7 months ago

No it does not. One of the main reasons is because it is much slower to compute. The second reason is that I must register a set of pre-defined functions in c++. Out of question to allow a user-defined function like in lidR it is too complex. Overall I'm keen to do it but it is not my priority.

I see, is it best to use this function for data with similar size trees then, and not amazon rain-forest data with extreme variance in canopy and heights? I should be able to use lasR for the normalisation and chm for the speed, and just have lidR handle the rest right? Also, I am yet to use the Derived metrics feature in lidR, but I assume I can get a .csv of all of the trees and their canopy size and height with it. Can I do the same with lasR?

This link in incorrect. Where did you find it? Thanks

Sure, here is a screenshot image

Jean-Romain commented 7 months ago

If you have any question, please open a dedicated thread in the discussion section