HenrikBengtsson / future.batchtools

:rocket: R package future.batchtools: A Future API for Parallel and Distributed Processing using batchtools
https://future.batchtools.futureverse.org
84 stars 9 forks source link

use batchtools directly for scheduling #70

Open myoung3 opened 3 years ago

myoung3 commented 3 years ago

Unless this has changed recently, future.batchtools creates a separate registry for each job. This approach misses out on some of the great features of batchtools--namely, being able to check on the status of all your jobs (including viewing logs, completion status, error messages etc) via a second interactive R session by navigating to the registry location. Since each job gets its own registry when using future.batchtools, this makes it very difficult to track the status of individual jobs.

Additionally, creating a separate registry for each job considerably slows down startup relative to a pure batchtools approach.