Describe the bug
When Artifact streaming is enabled on a Linux node in our Kubernetes cluster, we're experiencing problems with image pulls. Specifically, we're encountering "Failed to pull image" errors during deployments. Additionally, over time, the disk space on the node becomes filled up, leading to the eviction of all pods.
Observations:
With Artifact Streaming Enabled on node:
Failed image pulls during deployments.
Disk space gradually fills up over time.
All pods eventually get evicted due to the lack of available disk space.
With Artifact Streaming Disabled on node:
Deployments function as expected.
Images are pulled correctly without errors.
No significant disk space issues observed.
error:
Failed to pull image ".azurecr.io/products/api:master": rpc error: code = Canceled desc = failed to pull and unpack image ".azurecr.io/products/api:master": failed to resolve reference "**.azurecr.io/products/api:master": failed to do request: Head "https://localhost:8578/v2/products/api/manifests/master?ns=.azurecr.io": context cancel
To Reproduce
Steps to reproduce the behavior:
Enable artifact streaming on ACR
Enable Artifact Streaming on Node
Deploy pods with image from ACR
Expected behavior
Pods should be deployed
Screenshots
If applicable, add screenshots to help explain your problem.
Any relevant environment information
OS: Ubuntu 20.04
AKS / nodePools version is 1.28.3
Additional context
Add any other context about the problem here.
Describe the bug When Artifact streaming is enabled on a Linux node in our Kubernetes cluster, we're experiencing problems with image pulls. Specifically, we're encountering "Failed to pull image" errors during deployments. Additionally, over time, the disk space on the node becomes filled up, leading to the eviction of all pods.
Observations: With Artifact Streaming Enabled on node:
Failed image pulls during deployments. Disk space gradually fills up over time. All pods eventually get evicted due to the lack of available disk space.
With Artifact Streaming Disabled on node: Deployments function as expected. Images are pulled correctly without errors. No significant disk space issues observed.
error: Failed to pull image ".azurecr.io/products/api:master": rpc error: code = Canceled desc = failed to pull and unpack image ".azurecr.io/products/api:master": failed to resolve reference "**.azurecr.io/products/api:master": failed to do request: Head "https://localhost:8578/v2/products/api/manifests/master?ns=.azurecr.io": context cancel
To Reproduce Steps to reproduce the behavior:
Expected behavior Pods should be deployed
Screenshots If applicable, add screenshots to help explain your problem.
Any relevant environment information
Additional context Add any other context about the problem here.