r-lib / later

Schedule an R function or formula to run after a specified period of time.
https://r-lib.github.io/later
Other
137 stars 27 forks source link

Disable input handler when process is forked #141

Closed wch closed 3 years ago

wch commented 3 years ago

Closes #140.

This disables the input-handler-driven running of the event loop in the child process.

In the case where the processes don't call run_now() it's fine: functions scheduled by later() still exist in the child processes, but aren't executed. However, one problem is that the queue still exists in child processes. So if the child process calls run_now(), it will execute callbacks created in the parent process.

For example:

library(later)
library(future)

# Enable forked process
options(future.fork.enable = TRUE)
plan(multicore, workers = 2)

ff <- function(){
  message( Sys.getpid())
  later::later(ff, 2)
}
ff()

fs <- replicate(2, {
  future({
    for (i in 1:1000) {
      later::run_now(timeout = 1)
    }
  })
})

This will output something like:

18235
18241
18242
18235
18241
18242
...

In order to avoid that problem, it should also clear the callback registry when forked.

I'm inclined to just merge this now and then deal with the duplicated callback problem at another time, since it should be uncommon, and this fix will deal with the problem from #140. (And there's other things I need to focus on at the moment.)

There's also the issue of background threads (in the child process) being able to put things in the queue from the C API. I think that should just work, but I'm not 100% sure.

wch commented 3 years ago

I just came across this post on SO about the dangers of using fork with threads: https://stackoverflow.com/questions/6078712/is-it-safe-to-fork-from-within-a-thread/6079669#6079669

With future, the forking will always occur on the main R thread. The child processes will only have that main thread; other threads are not copied by a fork. However, if there are other threads and one of them happens to have locked a mutex when the fork happens, then that mutex will never be released in the child process and the child process will be deadlocked.

In later, a background thread will lock a mutex when it's adding a callback to the event loop, so a deadlocked child process can happen, though the probability is small.