Add `atm start`, `atm status` and `atm stop` commands

HDI-Project / ATM

Auto Tune Models - A multi-tenant, multi-data system for automated machine learning (model selection and tuning).

https://hdi-project.github.io/ATM/

MIT License

527 stars 141 forks source link

Add `atm start`, `atm status` and `atm stop` commands #130

Closed csala closed 5 years ago

csala commented 5 years ago

Add CLI commands to start ATM as a cluster.

Options should work as follows:

atm start: starts one or more workers in background, in wait mode
atm stop: stops all the workers
atm status: shows information about whether there is a cluster running or not and about how many workers it has.

The start command should start a master process using python-daemon, and this should use multiprocessing to spawn other workers as required. A pid file for the master process should be used to control whether the ATM is up or not.

Also, optionally, the start command should spawn a REST API app on a separated process.

beevabeeva commented 5 years ago

Awesome. I might be able to help with this. It's probably a good time to mention that I plan on basing my honours research on extending ATM to use parallelised GPU based ML algorithms, and run across an HPC cluster.

I'd first start with a naive approach: make a bash or Python script that would take in cluster run parameters, based on the example given by @csala in #128

csala commented 5 years ago

Awesome. I might be able to help with this. It's probably a good time to mention that I plan on basing my honours research on extending ATM to use parallelised GPU based ML algorithms, and run across an HPC cluster.

I'd first start with a naive approach: make a bash or Python script that would take in cluster run parameters, based on the example given by @csala in #128

Thanks for the help offer @beevabeeva! However this is currently being tackled by @pvk-developer in a python based approach, so I think it would be best if you wait for him to finish his development in order to avoid having multiple people working on the same.

On the other side the research around adding GPU based algorithms sound promising! Perhaps you'd like to open a separated issue to discuss the approach and align it with the future developments that are coming into ATM?

csala commented 5 years ago

Closed via #133

beevabeeva commented 5 years ago

Thanks for the help offer @beevabeeva! However this is currently being tackled by @pvk-developer in a python based approach, so I think it would be best if you wait for him to finish his development in order to avoid having multiple people working on the same.

That is great, thanks! On the other side the research around adding GPU based algorithms sound promising! Perhaps you'd like to open a separated issue to discuss the approach and align it with the future developments that are coming into ATM?

Will do. I am currently writing up the research proposal. It will be done on the 18th of May, at which point I will open a new issue with a link to the proposal pdf. Thanks for the support thus far!