NIEHS / chopin

Computation of Spatial Data by Hierarchical and Objective Partitioning of Inputs for Parallel Processing
https://niehs.github.io/chopin/
Other
10 stars 3 forks source link

Determine `sf`/`terra` class depending on future::plan() value #82

Closed sigmafelix closed 3 months ago

sigmafelix commented 3 months ago

plan(multicore) is available in nix systems (due to the support of fork-eval). With other than multicore plan, terra objects are not exportable to parallel workers. `par_functions should utilizeplanvalue to convert or to give a warning message whenterra` objects are put in.

sigmafelix commented 3 months ago

From 0.7.3, newly added internal functions detect whether a file path is vector or raster then par_grid convert SpatVector grid polygons into sf right before the future.apply::future_lapply is run to ensure the objects are exported to parallel workers. The approach still has caveats in file-based databases (i.e., GPKG) that can contain multiple layers either raster or vector. terra::vect attempts to read the first layer unless layer argument is specified, which avoids crashing the downstream functions. However, documentation should clearly state potential issues with file-based databases.