search

Mirantis / cri-dockerd

dockerd as a compliant Container Runtime Interface for Kubernetes
Apache License 2.0
1.01k stars 272 forks source link

Pulling image could fail when the image is large #372

Closed xinfengliu closed 2 weeks ago

xinfengliu commented 4 weeks ago

Expected Behavior

Pulling image mcr.microsoft.com/windows/servercore:ltsc2019 succeeds

Actual Behavior

Pulling image mcr.microsoft.com/windows/servercore:ltsc2019 fails.

This is because when there is no new progress for 1 minute, cri-dockred will abort the image pull. For a large image, the final step of "image extracting" could take more than 1 minute.

Steps to Reproduce the Problem

  1. Delete mcr.microsoft.com/windows/servercore:ltsc2019 image on target node win2019-1 if the image exists.

  2. Deploy this pod

    apiVersion: v1
kind: Pod
metadata:
name: ping
labels:
app: ping
spec:
containers:
- name: ping
image: mcr.microsoft.com/windows/servercore:ltsc2019 
imagePullPolicy: IfNotPresent
command: ["ping", "-t", "127.0.0.1"]
nodeSelector:
kubernetes.io/hostname: "win2019-1"

  3. Watch \k\cri-dockerd-mke\cri-dockerd-mke.log on node win2019-1

    ...
time="2024-06-02T09:42:31-07:00" level=info msg="Pulling image mcr.microsoft.com/windows/servercore:ltsc2019: 0dd0445527a5: Extracting [============================>                      ]  294.7MB/516.6MB"
time="2024-06-02T09:42:41-07:00" level=info msg="Pulling image mcr.microsoft.com/windows/servercore:ltsc2019: 0dd0445527a5: Extracting [=====================================>             ]  384.9MB/516.6MB"
time="2024-06-02T09:42:51-07:00" level=info msg="Pulling image mcr.microsoft.com/windows/servercore:ltsc2019: 0dd0445527a5: Extracting [=========================================>         ]  426.1MB/516.6MB"
time="2024-06-02T09:43:01-07:00" level=info msg="Pulling image mcr.microsoft.com/windows/servercore:ltsc2019: 0dd0445527a5: Extracting [============================================>      ]  464.6MB/516.6MB"
time="2024-06-02T09:43:11-07:00" level=info msg="Pulling image mcr.microsoft.com/windows/servercore:ltsc2019: 0dd0445527a5: Extracting [==================================================>]  516.6MB/516.6MB"
time="2024-06-02T09:43:21-07:00" level=info msg="Pulling image mcr.microsoft.com/windows/servercore:ltsc2019: 0dd0445527a5: Extracting [==================================================>]  516.6MB/516.6MB"
time="2024-06-02T09:43:31-07:00" level=info msg="Pulling image mcr.microsoft.com/windows/servercore:ltsc2019: 0dd0445527a5: Extracting [==================================================>]  516.6MB/516.6MB"
time="2024-06-02T09:43:41-07:00" level=info msg="Pulling image mcr.microsoft.com/windows/servercore:ltsc2019: 0dd0445527a5: Extracting [==================================================>]  516.6MB/516.6MB"
time="2024-06-02T09:43:51-07:00" level=info msg="Pulling image mcr.microsoft.com/windows/servercore:ltsc2019: 0dd0445527a5: Extracting [==================================================>]  516.6MB/516.6MB"
time="2024-06-02T09:44:01-07:00" level=info msg="Pulling image mcr.microsoft.com/windows/servercore:ltsc2019: 0dd0445527a5: Extracting [==================================================>]  516.6MB/516.6MB"
time="2024-06-02T09:44:11-07:00" level=error msg="Cancel pulling image mcr.microsoft.com/windows/servercore:ltsc2019 because it exceeded image pull deadline 1m0s. Latest progress 0dd0445527a5: Extracting [==================================================>]  516.6MB/516.6MB"

Specifications