Thanks for clustermq, I haven't used it yet but it looks very useful. I have written a package with similar goals in python called divvy.
It strikes me that it might be useful to have standard specification for back-end templates that could be used from either python or R. So, I have been looking for someone on the R side with a similar goal, and thought I'd reach out.
The idea is that you'd "configure your computing environment just once. You'd set up the template for SLURM or LFS or AWS Batch or whatever, and provide your settings like partition name, or whatever other variables you need. Then, we'd provide tools in both R and python that can use that standardized settings to submit jobs.
What I'm proposing is basically what you're already doing, except take the way you're storing the submission templates and settings and make it standard and independent of clustermq, so that one configuration can work for tools in various languages.
let me know if you're interested in pursuing that idea.
Thanks for clustermq, I haven't used it yet but it looks very useful. I have written a package with similar goals in python called divvy.
It strikes me that it might be useful to have standard specification for back-end templates that could be used from either python or R. So, I have been looking for someone on the R side with a similar goal, and thought I'd reach out.
The idea is that you'd "configure your computing environment just once. You'd set up the template for SLURM or LFS or AWS Batch or whatever, and provide your settings like partition name, or whatever other variables you need. Then, we'd provide tools in both R and python that can use that standardized settings to submit jobs.
What I'm proposing is basically what you're already doing, except take the way you're storing the submission templates and settings and make it standard and independent of clustermq, so that one configuration can work for tools in various languages.
let me know if you're interested in pursuing that idea.