Try to use a build and use small Triton image

[rbo]

At then moment, Triton image 10g big.

@goetzrieger please provide information to the small available version of triton

[rbo]

Right now, we are using:

• nvcr.io/nvidia/tritonserver:24.06-py3 7.64 GB

The minimal is a bit less

• nvcr.io/nvidia/tritonserver:24.06-py3-min 3.85 GB

Let's try to build an own only with onnx backend:

https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/customization_guide/compose.html

Backends: https://github.com/triton-inference-server/backend

python3 compose.py --backend onnxruntime --repoagent checksum --container-version 24.06

python3 compose.py --backend onnxruntime --repoagent checksum --container-version 24.06                             ─╯

/Volumes/Development/github.com/triton-inference-server/server/compose.py:275: SyntaxWarning: invalid escape sequence '\S'
  e0 = re.search("TRITON_SERVER_GPU_ENABLED=([\S]{1,}) ", vars)
/Volumes/Development/github.com/triton-inference-server/server/compose.py:288: SyntaxWarning: invalid escape sequence '\S'
  e = re.search("TRITON_SERVER_VERSION=([\S]{6,}) ", vars)
/Volumes/Development/github.com/triton-inference-server/server/compose.py:294: SyntaxWarning: invalid escape sequence '\S'
  e = re.search("NVIDIA_TRITON_SERVER_VERSION=([\S]{5,}) ", vars)
/Volumes/Development/github.com/triton-inference-server/server/compose.py:300: SyntaxWarning: invalid escape sequence '\S'
  dcgm_ver = re.search("DCGM_VERSION=([\S]{4,}) ", vars)
using container version 24.06
pulling container:nvcr.io/nvidia/tritonserver:24.06-py3
Traceback (most recent call last):
  File "/Volumes/Development/github.com/triton-inference-server/server/compose.py", line 516, in <module>
    argmap = create_argmap(images, FLAGS.skip_pull)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Volumes/Development/github.com/triton-inference-server/server/compose.py", line 218, in create_argmap
    p = subprocess.run(["docker", "pull", full_docker_image])
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py", line 1955, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'docker'

=> ln -s /opt/podman/bin/podman ~/bin/docker

[rbo]

PS.: We need a smaller image. I tried to deploy the triton server via GitOps on a new/empty robot. It failed:

Events:
  Type     Reason   Age                   From     Message
  ----     ------   ----                  ----     -------
  Warning  Failed   34m                   kubelet  Failed to pull image "nvcr.io/nvidia/tritonserver:24.06-py3": rpc error: code = Unknown desc = Error writing blob: error storing blob to file "/var/tmp/storage235593114/1": error happened during read: unexpected EOF
  Warning  Failed   8m20s (x8 over 3h9m)  kubelet  Error: ErrImagePull
  Warning  Failed   8m20s                 kubelet  Failed to pull image "nvcr.io/nvidia/tritonserver:24.06-py3": rpc error: code = Unknown desc = Error writing blob: error storing blob to file "/var/tmp/storage322025009/3": error happened during read: unexpected EOF
  Normal   Pulling  8m9s (x9 over 3h32m)  kubelet  Pulling image "nvcr.io/nvidia/tritonserver:24.06-py3"

[goetzrieger]

Build fails whatever I try, opened a Triton Server issue

https://github.com/triton-inference-server/server/issues/7513