search

bhanuc / nanonets

Demo API server
0 stars 0 forks source link

Running experiments in parallel using queue #1

Open prats226 opened 6 years ago

prats226 commented 6 years ago

https://github.com/bhanuc/nanonets/blob/aa65fe63b15a697a218bf63b6419be28e7952dae/app/workers/train.js#L22

Are all the experiments for a model running serially here? What modifications will you make if you want to convert worker into separate service?

bhanuc commented 6 years ago

Well currently we only have one workers, to make worker process in parallel, we just need to run the multiple worker script separately and it would process the ready jobs in parallel. The script is actually written without any dependence on the running app, so we can run it separately without any modifications actually. We can also make of cluster module to use all the cores for this purpose.