vatlab / sos

SoS workflow system for daily data analysis
http://vatlab.github.io/sos-docs
BSD 3-Clause "New" or "Revised" License
274 stars 45 forks source link

Setting up a test cluster environment #1252

Open BoPeng opened 5 years ago

BoPeng commented 5 years ago

We currently have a docker based "remote" host with a "task-spooler" mimicking a scheduler. This can be used to test base remote operations but not for any scenarios with decent number of tasks.

It can be helpful to set up a mini cluster using virutualbox/vmware from instructions herer

BoPeng commented 5 years ago

Using docker can also be an option: here