I would like to use packages {future.batchtools} and {furrr} to run loops in parallel on some HPC cluster.
In my loops, I save each result in some RDS file, and basically return nothing.
Is there any way to run future_walk() for this that would be non-blocking, i.e. once jobs for each element of the loop has been scheduled, then the R session becomes free again.
I would like to use packages {future.batchtools} and {furrr} to run loops in parallel on some HPC cluster. In my loops, I save each result in some RDS file, and basically return nothing. Is there any way to run
future_walk()
for this that would be non-blocking, i.e. once jobs for each element of the loop has been scheduled, then the R session becomes free again.