Docker management

[aarnphm]

Background

• Currently BentoML manages docker images via ./dev/release-docker-images.sh every docker directory contains a small release.sh to generate and push the docker images to the registry. This approach is pretty manual and a lot of repetitions are invoked in every release.sh.

• Additionally there aren't any ways for developer to develop Dockerfile and test the docker images locally. Developers has to go into each release.sh to comment out push commands.

Proposal

Project structure under docker/

.
├── generated #DO NOT EDIT 
│   └── {package}
│      └── {distro}{distro_version}
│               ├── Dockerfile   # base dockerfile
│               ├── runtime       # runtime dockerfile
│               ├── cudnn         # cuda enabled dockerfile
│               └── devel          # devel image with bentoml installed from git
├── templates
│   ├── alpine
│   ├── debian
│   ├── docs
│   └── rhel
├── tools
│   ├── bashrc
│   └── {package}
├── Dockerfile
├── manager.py
├── manifest.yml
├── pyproject.toml
└── README.md

• when install different version of conda → lack of testing → CI-related using project integration test. → Docker-in-docker or Sysbox
• supports multiple OS: amazonlinux - ubuntu - debian - alpine - centos
• deprecate bentoml/azure-functions

Implementation

Dependency will be separated from BentoML's main package → introduces the package as a side-car

We will render Dockerfile from jinja templates via manager.py.

Introduces manifest.yaml to control our release and supports for wider range of OS and architecture.

Inside our manifest.yaml:

---
# allows users to define custom registries
# this can also include GCR, NVCR, Heroku, etc.
repository:
  docker.io:
    only_if: HUB_PASSWORD
    user: HUB_USERNAME
    pwd: HUB_PASSWORD
    registry:
      model-server: docker.io/bentoml/model-server
      yatai-service: docker.io/bentoml/yatai-service

packages:
  model-server:
    devel: &default_specs
      - debian
      - centos
    runtime:
      - *default_specs
      - amazonlinux
      - alpine
    cudnn:
      - *default_specs
  yatai-service:
    runtime:
      - *default_specs

cuda_dependencies:
  11.3.1: &cuda11_3_1_deps
    cudart: 11.3.109-1
    cudnn8: 8.2.0.53-1
    libcublas: 11.5.1.109-1
    libcurand: 10.2.4.109-1
    libcusparse: 11.6.0.109-1
    libcufft: 10.4.2.109-1
    libcusolver: 11.1.2.109-1

# http://yaml.org/type/merge.html
release_spec: &tmpl_spec
  templates_dir: ""
  base_image: ""
  add_to_tags: ""
  multistage_image: True
  cuda:
    <<: *cuda11_3_1_deps

releases:
  debian:
    10:
      <<: *tmpl_spec
      templates_dir: templates/debian
      base_image: debian:buster-slim
      add_to_tags: "slim"
      cuda_prefix_url: "ubuntu2004"
  centos:
    8: &centos
      <<: *tmpl_spec
      templates_dir: templates/rhel
      base_image: centos:8
      add_to_tags: "centos8"
      cuda_prefix_url: "rhel8"
  amazonlinux:
    2: &no_cuda
      <<: *tmpl_spec
      templates_dir: templates/rhel
      base_image: amazonlinux:2
      add_to_tags: "ami2"
      cuda_prefix_url:
      cuda_requires:
      cuda:
  alpine:
    3.14:
      <<: *no_cuda
      templates_dir: templates/alpine
      base_image: alpine:3.14
      add_to_tags: "alpine"

Plan of Action

• [x] manager.py logics
• [x] create schema for docker images
• [ ] add CI integration test
• [ ] tests

Revision

• [x] create demo images for every new release.
• [x] update naming convention for new os

[aarnphm]

Done