r-lib / later

Schedule an R function or formula to run after a specified period of time.
https://r-lib.github.io/later
Other
137 stars 27 forks source link

Disabled when forked process is scheduled #140

Closed dipterix closed 3 years ago

dipterix commented 3 years ago

An example:

library(future)

# Enable forked process
options(future.fork.enable = TRUE)
plan(multicore, workers = 2)

ff <- function(){
  print(123)
  later::later(ff, 2)
}
ff()

# 2 workers, spawn 3 processes

fs <- replicate(3, {
  future({
    Sys.sleep(3)
  })
})

later::next_op_secs()
#> [1] -15.88667

I know forked process is discouraged, but under some situation, forked process can disable later from auto-run. Is there any suggestions on how should I re-instantiate global loop?

I tried later:::ensureInitialized(), but not worked.

dipterix commented 3 years ago

I guess this is an issue where the later autorun is shutdown by forked process, because when shutdown fork, later never gets resumed. This could potentially break the packages depending on later's auto-run feature because anyone can create pthreads using packages like parallel, future, or multicore and potentially disable the auto-run.

I found this trick used by fst to detect if forked process is spawned to avoid OpenMP from running. I was wondering how hard it will be to disable auto-run when a pthread is created?

In this way auto-run never gets shut down by a pthread.

wch commented 3 years ago

If I had to guess, I think that it's probably related to the file descriptors and input handler stuff in https://github.com/r-lib/later/blob/master/src/later_posix.cpp.