Closed nick-youngblut closed 1 year ago
At some point, there should be both a future
backend (via the package that you link), but also support for using workers as a cluster
object, which would both enable your use case.
However, I do unfortunately not have an ETA for this.
Solution to this will be a consequence of #119
It appears that tidymodels only supports parallelization via the
do*
packages. That includesdoFuture
so I was hoping that clustermq would be supported by future, but it appears that this package is all that exists, and it is just a rough developmental package.I really prefer using clustermq versus batchtools (thanks for making a great package!), but it looks like I'm going to have to go back to batchtools in order to parallelize tidymodels on a cluster.
Are there any plans to support tidymodels via future or another method?