e-sensing / sits

Satellite image time series in R
https://e-sensing.github.io/sitsbook/
GNU General Public License v2.0
470 stars 76 forks source link

sits_mosaic() multicores error #1220

Open paulacopelli opened 6 days ago

paulacopelli commented 6 days ago

Describe the bug
I encountered an error when using the sits_mosaic() function. Despite the layers being processed 100%, the final step of merging them was not completed. The error only occurs when using multiple cores.

To Reproduce
I ran the following code with 2 cores:

mosaic_dem <- sits_mosaic(
  cube       = cube_itata,
  crs        = "EPSG:32718",       # I tried with and without the crs
  roi        = roi_itata,
  multicores = 2,
  output_dir = paste0(raiz,"/out/cube_itata_mosaic"),
  version = "v3"
)

I received the following error message:

|=========================================================================================| 100% 
Error in purrr::map():
ℹ In index: 1.
Caused by error in .check_remote_errors():
! 1196 nodes produced errors; first error: Write error.
---
Backtrace:
     ▆
  1. ├─sits::sits_mosaic(...)
  2. │ └─sits:::.jobs_map_parallel_dfr(...)
  3. │   └─sits:::.jobs_map_parallel(jobs, fn, ..., progress = progress)
  4. │     ├─base::unlist(...)
  5. │     └─purrr::map(...)
  6. │       └─purrr:::map_("list", .x, .f, ..., .progress = .progress)
  7. │         ├─purrr:::with_indexed_errors(...)
  8. │         │ └─base::withCallingHandlers(...)
  9. │         ├─purrr:::call_with_cleanup(...)
 10. │         └─sits (local) .f(.x[[i]], ...)
 11. │           └─sits:::.parallel_map(round, fn, ..., progress = progress)
 12. │             └─sits:::.parallel_cluster_apply(x, fn, ..., pb = pb)
 13. │               └─parallel (local) .check_remote_errors(val)
 14. │                 └─base::stop(...)
 15. └─base::.handleSimpleError(...)
 16.   └─purrr (local) h(simpleError(msg, call))
 17.     └─cli::cli_abort(...)
 18.       └─rlang::abort(...)

However, running the same code with 1 core instead allowed the process to proceed:

mosaic_dem <- sits_mosaic(
  cube       = cube_itata,
  crs        = "EPSG:32718",
  roi        = roi_itata,
  multicores = 1,
  output_dir = paste0(raiz,"/out/cube_itata_mosaic"),
  version = "v3"
)

It skipped the already processed layers and successfully merged the mosaics.

If reporting a change from previous versions
Version: sits 1.5.1

Additional context
Running the mosaic with 1 core from the beginning also produces an error, but rerunning with 1 core after the initial failure works.

OldLipe commented 6 days ago

Dear @paulacopelli,

Thank you for your contribution and for providing a reproducible example.

We had the same behavior in some experiments that we did. We will investigate and return to you as soon as possible.

Thank you! Felipe Carvalho