Running Docker on RunPod doesn't work

[okdewit]

I used Skypilot docs and Mistral docs to create this YAML:

resources:
  accelerators: RTXA4000:1

run: |
  docker run --gpus all -p 8000:8000 ghcr.io/mistralai/mistral-src/vllm:latest \
                   --host 0.0.0.0 \
                   --model mistralai/Mistral-7B-Instruct-v0.2 \
                   --tensor-parallel-size 1

This spins up runpod/base:0.0.2 docker image, and I think it then tries run docker within that container, which won't work.

Is there a way to deploy docker images on RunPod through Skypilot, or maybe create a RunPod template through SkyPilot?

[7flash]

https://github.com/skypilot-org/skypilot/issues/3007

[kodxana]

@okdewit This is normal as RunPod pods are docker containers. You would need to turn it probably into template.

[romilbhardwaj]

RunPod does not allow running docker daemon in pods, so docker run and image_id: docker:.... through SkyPilot would not work.

As @7flash pointed out, #3007 is tracking support for custom templates, which may help with the image_id: docker:.... use case.