search

HenrikBengtsson / future.batchtools

:rocket: R package future.batchtools: A Future API for Parallel and Distributed Processing using batchtools
https://future.batchtools.futureverse.org
84 stars 9 forks source link

Request: add support for preemption #27

Open tantrev opened 6 years ago

tantrev commented 6 years ago

For schedulers like SLURM, preemptible jobs are really nice. Many University computing resources have a lot of spare computing power for preemptible jobs and having functionality from future.batchtools to support them (and re-schedule them if anything went wrong) would be fantastic.

HenrikBengtsson commented 6 years ago

Yes, I agree, that would be very useful.

Being able to relaunch futures, in general, is a feature request that others asked about. I've started a specific issue on this over at https://github.com/HenrikBengtsson/future/issues/205.

Do you know if the underlying batchtools package provides mechanisms for working with preemptible jobs? I think that would be needed/very useful in order to start thinking about how this can be supported. It'll also help in identifying the API for https://github.com/HenrikBengtsson/future/issues/205.