search

canonical / data-science-stack

Stack with machine learning tools needed for local development.
Apache License 2.0
17 stars 7 forks source link

Explore: How to run shell script or python on snap install #3

Closed misohu closed 10 months ago

misohu commented 11 months ago

Why it needs to get done

For shipping the the DSS we want to run shell/python script to setup DSS.

What needs to get done

shell script/python file needs to be executed on 

sudo snap intall dss --classic

When is the task considered done

We know how to execute shell script/python script on snap install (if it is possible).

misohu commented 11 months ago

We have created first POC repo to deploy the DSS. There are three main use cases we need to solve with the deployment of DSS.

Following are the main problems we found during the eploration of snap solution for deployment of DSS

Problems

  1. Snap problem: Isolation

Every strict snap is running in the isolated environment. Meaning there is no way to install snaps from other snaps. We cannot just. 

sudo snap install yq
sudo snap install juju-wait --classic
sudo snap install juju --classic --channel=3.1/stable
sudo snap install microk8s --classic --channel=1.26/stable

This problem will also bite us during removal phase when we need to remove installed snaps.

  1. Snap problem: Using other snaps

Because of the aforementioned fact that the snaps are running in isolated environment, we cannot simply use executables installed by other charms. e.g.

Work arounds for snap

  1. Isolation To overcome this problem we can let the dss snap (our snap) output the bash script which will be applied on the machine to install requirements and do the setup which is not possible form inside snap. Example form sunbeam which returns prepare machine script which can be applied by bash. In our case the script will provide bash code for installing requirements and setting up microk8s addons. e.g.: 
sudo snap install yq
sudo snap install juju-wait --classic
sudo snap install juju --classic --channel=3.1/stable
sudo snap install microk8s --classic --channel=1.26/stable

sudo usermod -a -G microk8s ubuntu
sudo mkdir /home/ubuntu/.kube
sudo mkdir -p /home/ubuntu/.local/share
sudo chown -f -R ubuntu /home/ubuntu/.kube
sudo chown -f -R ubuntu /home/ubuntu/.local/share

sudo microk8s enable dns storage metallb:"10.64.140.43-10.64.140.49,192.168.0.105-192.168.0.111"
sleep 30
sudo microk8s.kubectl wait --for=condition=available -nkube-system deployment/coredns deployment/hostpath-provisioner
sudo microk8s.kubectl -n kube-system rollout status ds/calico-node
  1. Using other snaps If we want to use lets say juju or kubectl inside our snap we need to install those in the snap. We can have a juju binary packed inside a snap and we can pass the microk8s config to the snap which will connect to the microk8s running locally. Having these in the snap we can setup juju controller and model from iside the snap and deploy bundle/charms.

Here is how the exampel workflow would look like for the snap workaround:

# install
sudo snap install dss --channel 0.1/stable
dss setup-env | sudo bash -x

# interact
dss list-notebooks
dss create-notebook kimonas --image abs:123 --gpu

# cleanup
dss cleanup-env | sudo bash -x
sudo snap unintstall dss --purge

# upgrade
sudo snap refresh dss --channel 0.2/stable
dss update-env | sudo bash -x

Usefull links for building the snap: