search

mlhubber / mlhub

Machine Learning Model Repository Hub Command Line Tool.
MIT License
20 stars 12 forks source link

Move to containers/AKS for demos #10

Open gjwgit opened 6 years ago

gjwgit commented 6 years ago

This is a sizeable task for someone to build their experience of containers and Azure Kubenete Services

Consider whether we have a single mlhub container fully stacked with all that is needed. A user could download this once and all mlhub packages run in this container. Or can we host a container on AKS?

We could have a single mlhub image running Ubuntu, Python3, R, mlhub. Build it from the official ubuntu image (70MB)

gjwgit commented 6 years ago

There are alternatives too. A container is quite a sizeable object (e.g., 1GB). The concept of SNAPs may also be a contender, though it targets Linux. Nonetheless, even Microsoft (e.g., Skype) seem to be getting in on the SNAP concept for Linux packaging.

gjwgit commented 6 years ago

Alternative may be to simply mimic virtual environment with local installs within the model folder. Use R_LIBS to affect R's search path. Similarly for Python

simonzhaoms commented 5 years ago

Another scenario for container: It is slow and time-consuming to create a new DLVM on Azure for testing a model package extensively from scratch without any dependencies pre-installed. You have to setup username, password, SSH key, and install mlhub, though all these can be automated by Azure script. But it may a lot easier to clone a container image locally within the DLVM with all already setting up for test a model package.

simonzhaoms commented 5 years ago

I collect a list of possible solutions at https://github.com/simonzhaoms/pkgdep which is still in progress and will be updated as I learn more.

simonzhaoms commented 5 years ago

A fresh installation of mlhub takes 4 minutes and 26 seconds. And docker pull kayon/mlhub takes 5 minutes.