Azure / kubernetes-keyvault-flexvol

Azure keyvault integration with Kubernetes via a Flex Volume
MIT License
253 stars 83 forks source link

Unable to pull image from Microsoft azure container registry - NSG Rules #167

Closed bandarusrinivas closed 4 years ago

bandarusrinivas commented 4 years ago

Describe the bug

The team has deployed Azure pod identity which uses pods from Microsoft Container Registry. They use service tags to allow the outbound network traffic. The MicrosoftContainerRegistry tag was applied to an outbound network security rule for port 443. The team is getting the following event in their AKS cluster when trying to pull the image:

The IP address 204.79.197.219 is not in the ServiceTags revision published 2020-01-06. flexvol open source project try to use 204.79.197.219 https://www.microsoft.com/en-us/download/details.aspx?id=56519 Why this IP doesn't allow to pull image?

5s Warning Failed pod/nmi-md8sn Failed to pull image "mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.5.2": rpc error: code = Unknown desc =error pulling image configuration: Get https://mcreus0.cdn.mscr.io///docker/registry/v2/blobs/sha256/cd/cd3e1d35806b542085d4b9d0e8f830eec1229185/data?P1=6876786&P2=1&P3=1&P4=qxH0VXk6%2BvJBk7BliR9oDxkrYPN%2FfS01-09T20%3A43%3A11Z&sig=BHsYWp8RKTIVseBJDU1saheHyrpxWCbw%3D&sp=r&sr=b&sv=2016-05-31&regid=aba285c624a04409823bhfjkghfdkj945749: dial tcp 204.79.197.219:443: i/o timeout

Steps To Reproduce

internal-network/nsg_outbound.tf file snippet resource "azurerm_network_security_rule" "allow_mcr_outbound" { name = "AllowMCRAD" resource_group_name = var.common_rg_name network_security_group_name = azurerm_network_security_group.network_security_group.name description = "Allows pull from MCR." protocol = "" source_port_range = "" destination_port_range = "443" source_address_prefix = "VirtualNetwork" destination_address_prefix = "MicrosoftContainerRegistry.EastUS" access = "Allow" priority = "4005" direction = "Outbound" }

Expected behavior Should be able to pull images

Key Vault FlexVolume version

Access mode: service principal or pod identity pod identity

Kubernetes version kubernetes_version = "1.14.8" TERRAFORM_VERSION=0.12.9 KUBECTL_VERSION=1.14.2 AZURECLI_VERSION=2.0.74 apiVersion: "aadpodidentity.k8s.io/v1"

Additional context

bandarusrinivas commented 4 years ago

Closing this issue as this is not related to the flex volume, will check with Microsoft azure network team.