RFE - Support GPUs

[nnachefski]

Working with OCP 3.10, i am able to get the nvidia device plugin working in my project. This allows me to deploy the standard tensorflow+jupyter image on my GPUs (via Openshift).

I would like the jupyterhub s2i image for Openshift to have this same capability.

-Nick

[GrahamDumpleton]

My knowledge of using the GPU cards is very limited. Have you done all the host setup requirements for nodes which OpenShift is running on?

• https://blog.openshift.com/use-gpus-with-device-plugin-in-openshift-3-9/

As to injecting the environment variables and resource limit field for the NIVIDA GPU, that needs to be done via the JupyterHub config, by adding in appropriate Python code snippets to it to set things up. Or if using the preferred method of configuring a custom JupyterHub deployment, through a S2I build step for the custom JupyterHub image.

FWIW, for a POC where want to the S2I build mechanism for customising JupyterHub (since not everything can be done through the config map), you might look at:

• https://github.com/jupyter-on-openshift/poc-hub-keycloak-auth

To deploy that example all you need do is run the single step of:

oc new-app https://raw.githubusercontent.com/jupyter-on-openshift/poc-hub-keycloak-auth/master/templates/jupyterhub.json

As well as deploying JupyterHub, it also deploys KeyCloak with authentication enabled out of the box for self registration. Once up, you can go into KeyCloak as admin and change the realm config to use a separate authentication provider as necessary.

The example is easier to get going, but because it shows authentication can be more of interest as shows how you can protect access.

Either way, once up and running, to quickly try adding GPU support, edit the config map for JupyterHub (usually jupyterhub-cfg), and to the jupyterhub_config.py entry add:

c.Spawner.environment.update(dict(
       NVIDIA_VISIBLE_DEVICES='all',
       NVIDIA_DRIVER_CAPABILITIES='compute,utility',
       NVIDIA_REQUIRE_CUDA='cuda>=8.0'))

Also add:

c.KubeSpawner.extra_resource_limits = {'nvidia.com/gpu': '1'}

These are added as Python code, so ensure start of first line of each configuration setting is not indented.

Once the config map is changed, start a new deployment of JupyterHub. Shutdown any existing Jupyter notebook instance and start a new one so that it picks up the new config.

[GrahamDumpleton]

Going to close this. The repo:

• https://github.com/jupyter-on-openshift/poc-hub-tensorflow-gpu

was created sometime back to demonstrate GPU usage, although have never got feedback to confirm it all works okay. I didn't have the hardware to verify it.