search

jupyter / docker-stacks

Ready-to-run Docker images containing Jupyter applications
https://jupyter-docker-stacks.readthedocs.io
Other
7.83k stars 2.98k forks source link

Build GPU Variants of Current Images #1557

Open sdwalker62 opened 2 years ago

sdwalker62 commented 2 years ago

Hello everyone! First, I just want to say that at work we almost exclusively use Jupyter as our data science platform and appreciate everything that the community and the Jupyter team has done to create such a wonderful tool!

I have searched through the current list of issues and pull requests open in this repository and have not found and that match this request so I am starting a new one. Apologies if I have missed any. Our work necessitates the use of cuda-enabled containers as we primarily train deep learning models. We have made a slight modification to the makefile that allows us to re-use most of the infrastructure already in place to build the various levels of notebooks.

I am aware of the fantastic work being done at https://github.com/iot-salzburg/gpu-jupyter/ and suggest anyone who needs gpu-enabled containers to check it out as a first stop. This solution will work for many, and it contains some additional features which are nice to have, but it is slightly different than what we prepose.

The current docker-stacks starts with the base-notebook which is built from an Ubuntu 20.04 layer. Our preposal is to build a separate set of notebooks based on the nvidia/cuda:11.3.0-cudnn8-runtime-ubuntu20.04 image (we chose this for compatibility with Pytorch). This is similar to what the iot-salzburg repository does in their images but maintains the minimalism found in some of the base notebooks for those who are looking for a solid foundation for their own custom images. With the change to the makefiles it would be easy to build two sets of images: one for the standard notebooks as they exist now, and another for the cuda-enabled variants.

To address the selection criteria found in the documention for new features:

I would be happy to answer any questions about this proposal!

mathbunnyru commented 2 years ago

@sdwalker62 first of all, thank you for your suggestion! I'm sorry it took such a long time to answer you.

I see a growing interest in the GPU-related docker stacks because people ask more and more about them.

But I think right now it's not easy to add a new image. And in this case a whole set of new images. The main reason is our complicated and slow build system.

The build times are identical to the current build times as only the base layers changes.

Right now, if we add new images, this will add time to our workflow. Please take a look at our proposal and what our build system might look like in the future. https://github.com/jupyter/docker-stacks/issues/1407

The complexity of this change is low. We have a simply python script that replaces lines in the base-notebook and a new section in the Makefile that is minimal and could be hooked into the current build process

I think you can actually get rid of this patching because we have a way to pass custom arguments to docker build like this: DOCKER_BUILD_ARGS='--build-arg ROOT_CONTAINER="nvidia/cuda:12.3.1-devel-ubuntu22.04"' make build/docker-stacks-foundation. I think it builds the image you wanted it to build.

Please, tell me, if this helps.

Update 2024.01.17: fixed the command to use more recent image and docker-stacks-foundation image (which is the root image now).

mathbunnyru commented 2 years ago

This works because we have ARG ROOT_CONTAINER in our docker-stacks-foundation image. https://github.com/jupyter/docker-stacks/blob/main/images/docker-stacks-foundation/Dockerfile#L6

mathbunnyru commented 2 years ago

@sdwalker62 if you're still interested in this issue - we've changed our build system completely, so it should probable be easier now to add cuda based images.

markusschmaus commented 1 year ago

It would be awesome if such cuda based images were available.

mathbunnyru commented 1 year ago

I think it's actually currently possible to have cuda based images and not increase build time at all. If someone wants to do it, please, take a look here, this is where you will need to add new steps: https://github.com/jupyter/docker-stacks/blob/main/.github/workflows/docker.yml

benz0li commented 8 months ago

@sdwalker62 You may be interested in my/b-data's CUDA-enabled JupyterLab Python docker stack:

[...]

Similar projects

What makes this project different:

  1. Multi-arch: linux/amd64, linux/arm64/v8
  2. Derived from nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04
    • including development libraries and headers
  3. TensortRT and TensorRT plugin libraries
    • including development libraries and headers
  4. IDE: code-server next to JupyterLab
  5. Just Python – no Conda / Mamba

See Notes for tweaks, settings, etc.

P.S.: I have whitelisted your GitHub account at https://demo.cuda.jupyter.b-data.ch so you may quickly test online.

mathbunnyru commented 5 months ago

I updated some advices in my comments above, so they work with current set of images.

If someone comes across this issue and feels they need GPU images, please, upvote this issue - this will make it more clear for me and other maintainers, that community is really interested in such images. Also, I would highly appreciate the information on how to choose the relevant nvidia/cuda base image - there are many of them and I'm not sure which version community needs and wants the most.

mathbunnyru commented 4 months ago

New CUDA enabled pytorch-notebook images are pushed and ready to use 🎉 https://quay.io/repository/jupyter/pytorch-notebook?tab=tags

fzyzcjy commented 3 months ago

Hi, I am interested in having docker image for jupyter + pytorch + gpu, because docker is quite convenient and deep learning really needs GPU.

Therefore, I wonder is there any shortcomings when using these? I would love to hear from you experts!

mathbunnyru commented 3 months ago

@sdwalker62 @KopfKrieg @joglekara @TylerSpears @romainrossi @ChristofKaufmann @fzyzcjy @mfreeman451 @markusschmaus @benz0li Now we have cuda-enabled variants of jupyter/pytorch-notebook and jupyter/tensorflow-notebook. You can find more information here: https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#cuda-enabled-variant.

These images are still based on top of regular Ubuntu images, but install GPU versions of pytorch or tensorflow. I am not sure builds on top of Nvidia Ubuntu images are possible (because of the licensing). If you need such images, it would be nice to know the reasons and the use cases though.

I would like to thank @johanna-reiml-hpi for implementing general variant concept and making it work for pytorch (https://github.com/jupyter/docker-stacks/pull/2091) and @ChristofKaufmann for making it work for tensorflow (https://github.com/jupyter/docker-stacks/pull/2100).

yuvipanda commented 3 months ago

@mathbunnyru do you think it would be a useful idea to make a blog post on the Jupyter blog announcing these?

benz0li commented 3 months ago

I am not sure builds on top of Nvidia Ubuntu images are possible (because of the licensing).

Everything branded NVIDIA and/or CUDA comes with proprietary licenses.

The nvidia/cuda images entail the NVIDIA Deep Learning Container License.

Every nvidia-* Python package entails its own NVIDIA and/or CUDA license terms.

Whether or not you are aware of it, you agree to their End User License Agreement (EULA).

fzyzcjy commented 3 months ago

Thanks for the image!

If you need such images, it would be nice to know the reasons and the use cases though.

My two cents: Have not tried it yet, but IIRC some pip packages require proper cuda environments to compile some C++ code that is generated on the fly to maximize performance. (Maybe https://github.com/microsoft/DeepSpeed or some other lib, I do not remember very clearly)

Whether or not you are aware of it, you agree to their End User License Agreement (EULA).

Wondering whether it is possible to mark jupyter's corresponding images with that license, while other image with the original normal license

mathbunnyru commented 3 months ago

@mathbunnyru do you think it would be a useful idea to make a blog post on the Jupyter blog announcing these?

Thanks for the suggestion. I’ll do it!

benz0li commented 3 months ago

I am not sure builds on top of Nvidia Ubuntu images are possible (because of the licensing).

Cross reference: Uploading of container developed on top of nvidia/cuda images (#224) · Issues · nvidia / container-images / cuda · GitLab

mathbunnyru commented 3 months ago

@mathbunnyru do you think it would be a useful idea to make a blog post on the Jupyter blog announcing these?

Thanks for the suggestion. I’ll do it!

Sent for a review.

mathbunnyru commented 3 months ago

@benz0li @yuvipanda could you please tell me, what do I need to do to make GPU work? Simply add --gpus all when running docker image or is there something else?

I don't use GPU versions of images, so I don't have any hands-on experience, sorry for that.

benz0li commented 2 months ago

@benz0li @yuvipanda could you please tell me, what do I need to do to make GPU work?

NVIDIA GPU + NVIDIA Linux driver + NVIDIA Container Toolkit

ℹ️ The host running the GPU accelerated images only requires the NVIDIA driver, the CUDA toolkit does not have to be installed.

Prerequisites: See https://github.com/b-data/jupyterlab-python-docker-stack/blob/83e8c3b830db83b8c73457487605f44be9e4e487/CUDA.md#prerequisites

Simply add --gpus all when running docker image or is there something else?

Yes.

The above information is for Linux hosts. If you are working on a Windows host:

ℹ️ Current Apple hardware: No NVIDIA GPU = No CUDA support.

ChristofKaufmann commented 2 months ago

According to the nvidia doc: When setting the environment variable ENV NVIDIA_VISIBLE_DEVICES all as specified in the CUDA variants, you don't need to specify it on the command line. Haven't tested it though.

benz0li commented 2 months ago

According to the nvidia doc: When setting the environment variable ENV NVIDIA_VISIBLE_DEVICES all as specified in the CUDA variants, you don't need to specify it on the command line. Haven't tested it though.

@ChristofKaufmann This only works if either

or

ℹ️ Specifying --gpus in the CLI automatically invokes the nvidia runtime.

ChristofKaufmann commented 2 months ago

Thanks for the clarification!

mathbunnyru commented 2 months ago

@benz0li @yuvipanda could you please tell me, what do I need to do to make GPU work?

NVIDIA GPU + NVIDIA Linux driver + NVIDIA Container Toolkit

ℹ️ The host running the GPU accelerated images only requires the NVIDIA driver, the CUDA toolkit does not have to be installed.

Prerequisites: See https://github.com/b-data/jupyterlab-python-docker-stack/blob/83e8c3b830db83b8c73457487605f44be9e4e487/CUDA.md#prerequisites

Simply add --gpus all when running docker image or is there something else?

Yes.

  • Docker: --gpus all or --gpus '"device=all"'
  • Podman: --device 'nvidia.com/gpu=all'

The above information is for Linux hosts. If you are working on a Windows host:

ℹ️ Current Apple hardware: No NVIDIA GPU = No CUDA support.

Thanks, @benz0li 👍

mathbunnyru commented 2 months ago

Posted 🎉 https://blog.jupyter.org/cuda-enabled-jupyter-docker-images-8a9f8b8f2158