search

HenrikBengtsson / future.BatchJobs

:rocket: R package: future.BatchJobs: A Future API for Parallel and Distributed Processing using BatchJobs [Intentionally archived on CRAN on 2021-01-08]
https://cran.r-project.org/package=future.BatchJobs
8 stars 0 forks source link

Is support for BatchTools planned? #69

Closed alexvorobiev closed 7 years ago

alexvorobiev commented 8 years ago

Now that BatchTools have been released and declared a successor to BatchJobs, is future.BatchTools planned? The advantage I am really looking forward to is "Ongoing troubles with the data base and the file system. " - of which I have seen plenty.

HenrikBengtsson commented 8 years ago

That was pleasant news. Yes, the future will be bright also there. I promise. But it might be a while:

https://github.com/HenrikBengtsson/future.batchtools

andypohl commented 7 years ago

That is good news. I am looking into adding a new batch scheduler (HTCondor) to batchtools, and it'll be nice to have those great constructs from future and foreach. I like the hierarchy of the abstraction you've designed. It's very elegant, and having a non-blocking foreach is really good, especially if you're working interactively on the R console. I looked at the issues in batchtools and it seems HTCondor is in the plans. So I'll probably have my attention there for some time. Good luck porting this to future.batchtools. I hope it goes smooth.

HenrikBengtsson commented 7 years ago

Hi folks, I'm happy to announce that future.batchtools 0.3.0 is nowavailable at https://github.com/HenrikBengtsson/future.batchtools.

I've tested it on our local TORQUE / PBS system and it got significantly improved after I reviewed the JOSS batchtools: Tools for R to work on batch systems article.

I'll hold back on a CRAN submission for a bit, because I suspect there will be some immediate issues reported. I already have a few things in mind that I think should be added / is missing.

Please move over to https://github.com/HenrikBengtsson/future.batchtools for futher discussion on that package. All types of feedback are welcome.

alexvorobiev commented 7 years ago

Excellent news! Thank you!