Kubernetes support

[sverhoeven]

When grpc4bmi is running on a Kubernetes cluster it would be nice to start a model using the native Kubernetes APIs.

Pangeo configured their environment so from a Juptyer notebook a Dask cluster can be launched with http://kubernetes.dask.org

[sverhoeven]

The model container should have a volume that is shared between it and the notebook container, so that the model config file and input files can be staged in the notebook container and used on the model container.

JupyterHub on Kubernetes use a readwriteonce volume for each user and additional volumes can be added.

I see several options:

• Add the model container to the pod of the notebook container and using https://kubernetes.io/docs/tasks/access-application-cluster/communicate-containers-same-pod-shared-volume/
• Make the personal user storage volume available on the model container by changing it to readwritemany and attach it to the model container
• Attach an extra readwritemany volume to both the notebook and model container
• Share no volumes between the notebook and model container. Somehow the config file and input datafiles must be staged on the model container.

[sverhoeven]

• notebook can write to a volume and model containers have volume in read only mode.

[sverhoeven]

• notebook can write before model.setup(), after volume unmounted from notebook container and mounted on model container
• the notebook container could start a file server like samba or minio. The model container should mount that file server in a sidecar. Con the model container needs the notebook container and the files are exported/mounted twice which can be less performant