I would like to pass large datasets that are stored as ff files (which map memory to storage) but I have been encountering an issue with using the future.callr package. I was wondering if there is a way to overcome this issue?
`
library(future)
plan(future.callr::callr, workers = 8L)
x <- ff::ff(1:100)
y %<-% { x[sample(100,10)]}
y
Error: callr subprocess failed: could not start R, exited with non-zero status, has crashed or was killed
`
I would like to pass large datasets that are stored as ff files (which map memory to storage) but I have been encountering an issue with using the future.callr package. I was wondering if there is a way to overcome this issue? `