search

r-lidar / lasR

Fast and Pipable Airborne LiDAR Data Tools
https://r-lidar.github.io/lasR/
GNU General Public License v3.0
43 stars 0 forks source link

pipeline gets processed for no output if path does not exist #54

Closed wiesehahn closed 2 weeks ago

wiesehahn commented 2 weeks ago

When running a pipeline which writes files to disk but the folder does not exist it processes the entire pipeline (including spatial indexing if not present) and returns an error at the end. E.g. with

exec(dtm(1, ofile = "C:/path_to_folder/*_dtm.tif"), on = f)

the error is returned for each input file

ERROR 4: Attempt to create new tiff file `C:\path_to_folder\filename_dtm.tif' failed: No such file or directory

I think it would make sense to evaluate this before starting the process.

Jean-Romain commented 2 weeks ago

Sure. Even better the path could be created like in lidR. Thank you for reporting.

Jean-Romain commented 2 weeks ago

I'm not able to reproduce. It fails instantaneously as expected. Tested with 63 larges files. Can you describe the behavior you are observing with more details.

wiesehahn commented 2 weeks ago

Ah ok, you are right. Usually it fails instantaneously. So in my case I guess the indexing happened anyway (no matter if present or not, no matter if outdirectory is present or not). This might be what took long before failing.

library(lasR)
set_parallel_strategy(concurrent_files(4))
exec(dtm(5, ofile = "C:/Users/jwiesehahn/somenonexistentfolder/*_dtm.tif"), on = "C:/Users/jwiesehahn/data/sample.vpc")
#> 4 files do not have a spatial index. Spatial indexing speeds up tile buffering and spatial queries drastically.
#> Files will be indexed on-the-fly. This will take some extra time now but will speed up everything later.
#> Error: in rasterize while initalizing file: error 4 while creating GDAL dataset. Attempt to create new tiff file `C:\Users\jwiesehahn\somenonexistentfolder\las_dtm.tif' failed: No such file or directory
Jean-Romain commented 2 weeks ago

This indeed very likely related to write_lax. Probably not exactly the way you think. I think the first file and its 8 neighbors were indexed and then it failed. If you processed in parallel maybe more files were indexed before to fail.