Workers keep running if the master is a callr process and terminates early.

mschubert / clustermq

R package to send function calls as jobs on LSF, SGE, Slurm, PBS/Torque, or each via SSH

https://mschubert.github.io/clustermq/

Apache License 2.0

146 stars 27 forks source link

Workers keep running if the master is a callr process and terminates early. #130

Closed wlandau closed 5 years ago

wlandau commented 5 years ago

Ordinarily, clustermq automatically terminates workers if the master terminates, which is extremely helpful. However, if the master is a callr process, then workers keep running even after SIGINT. Example:

callr::r(function() clustermq::Q(Sys.sleep, 60, n_jobs = 1))

After I terminate this call early with CTRL-C, the worker stays up on SGE.

Related: https://github.com/ropensci/drake/issues/772. cc @pat-s.

mschubert commented 5 years ago

This seems to be related to callr rather than clustermq:

> callr::r(function() { on.exit(system("echo here > ~/test.out")); Sys.sleep(0) })
> callr::r(function() { on.exit(system("echo here2 > ~/test2.out")); Sys.sleep(60) }) ^C

$ cat test.out
here
$ cat test2.out
cat: test2.out: No such file or directory

Nothing I can do about that.