search

kubernetes-sigs / external-dns

Configure external DNS servers (AWS Route53, Google CloudDNS and others) for Kubernetes Ingresses and Services
Apache License 2.0
7.73k stars 2.57k forks source link

Microsoft Azure Gov Cloud Causing SubscriptionNotFound Error #1457

Closed flyer299 closed 4 years ago

flyer299 commented 4 years ago

What happened: Installed External DNS in the Microsoft Azure Government Cloud (VA) on AKS for K8S results in an error message: SubscriptionNotFound

It looks like it is using the incorrect path to retrieve the Subscription information from Azure when in the Gov Cloud.

Log Files from the External-DNS Pod

time="2020-03-04T16:54:33Z" level=info msg="config: {Master: KubeConfig: RequestTimeout:30s IstioIngressGatewayServices:[] ContourLoadBalancerService:heptio-contour/contour Sources:[ingress] Namespace: AnnotationFilter: FQDNTemplate: CombineFQDNAndAnnotation:false IgnoreHostnameAnnotation:false Compatibility: PublishInternal:false PublishHostIP:false ConnectorSourceServer:localhost:8080 Provider:azure GoogleProject: GoogleBatchChangeSize:1000 GoogleBatchChangeInterval:1s DomainFilter:[] ExcludeDomains:[] ZoneIDFilter:[] AlibabaCloudConfigFile:/etc/kubernetes/alibaba-cloud.json AlibabaCloudZoneType: AWSZoneType: AWSZoneTagFilter:[] AWSAssumeRole: AWSBatchChangeSize:1000 AWSBatchChangeInterval:1s AWSEvaluateTargetHealth:true AWSAPIRetries:3 AWSPreferCNAME:false AzureConfigFile:/etc/kubernetes/azure.json AzureResourceGroup:NXT-SW AzureSubscriptionID: 
AzureUserAssignedIdentityClientID: CloudflareProxied:false CloudflareZonesPerPage:50 CoreDNSPrefix:/skydns/ RcodezeroTXTEncrypt:false InfobloxGridHost: InfobloxWapiPort:443 InfobloxWapiUsername:admin InfobloxWapiPassword: InfobloxWapiVersion:2.3.1 InfobloxSSLVerify:true InfobloxView: InfobloxMaxResults:0 DynCustomerName: DynUsername: DynPassword: DynMinTTLSeconds:0 OCIConfigFile:/etc/kubernetes/oci.yaml InMemoryZones:[] PDNSServer:http://localhost:8081 PDNSAPIKey: PDNSTLSEnabled:false TLSCA: TLSClientCert: TLSClientCertKey: Policy:sync Registry:txt TXTOwnerID:default TXTPrefix: Interval:1m0s Once:false DryRun:false LogFormat:text MetricsAddress::7979 LogLevel:info TXTCacheInterval:0s ExoscaleEndpoint:https://api.exoscale.ch/dns ExoscaleAPIKey: ExoscaleAPISecret: CRDSourceAPIVersion:externaldns.k8s.io/v1alpha1 CRDSourceKind:DNSEndpoint ServiceTypeFilter:[] CFAPIEndpoint: CFUsername: CFPassword: RFC2136Host: RFC2136Port:0 RFC2136Zone: RFC2136Insecure:false RFC2136TSIGKeyName: RFC2136TSIGSecret: RFC2136TSIGSecretAlg: RFC2136TAXFR:false NS1Endpoint: NS1IgnoreSSL:false TransIPAccountName: TransIPPrivateKeyFile:}"
time="2020-03-04T16:54:33Z" level=info msg="Instantiating new Kubernetes client"
time="2020-03-04T16:54:33Z" level=info msg="Using inCluster-config based on serviceaccount-token"
time="2020-03-04T16:54:33Z" level=info msg="Created Kubernetes client https://10.2.0.1:443"
time="2020-03-04T16:54:34Z" level=info msg="Using client_id+client_secret to retrieve access token for Azure API."
time="2020-03-04T16:57:01Z" level=error msg="dns.ZonesClient#ListByResourceGroup: Failure sending request: StatusCode=0 -- Original Error: Get https://management.azure.com/subscriptions/fad8470a-e1a9-4209-b526-011761c080a0/resourceGroups/NXT-SW/providers/Microsoft.Network/dnsZones?api-version=2018-05-01: read tcp 10.0.0.32:37126->20.44.16.113:443: read: connection reset by peer"
time="2020-03-04T16:57:01Z" level=error msg="dns.ZonesClient#ListByResourceGroup: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code=\"SubscriptionNotFound\" Message=\"The subscription 'fad8470a-e1a9-4209-b526-011761c080a0' could not be found.\""

Notice the URL path to get Subscription information is: https://management.azure.com/subscriptions/fad8470a-e1a9-4209-b526-011761c080a0/resourceGroups/NXT-SW/providers/Microsoft.Network/dnsZones?api-version=2018-05-01

When operating in the Government cloud the URL Path should be azure.us and not azure.com (plus I think there are some other differences).

What you expected to happen: I expect there to be a configuration option to enable Government Cloud Support so that the project will use the correct URL Paths for the Government Cloud.

How to reproduce it (as minimally and precisely as possible): 1) Follow the instructions Here: https://github.com/kubernetes-sigs/external-dns/blob/master/docs/tutorials/azure.md 2) Perform it in the Azure Government Cloud in VA

Anything else we need to know?: Every other project we have attempted to get working in the Gov Cloud required a configuration option to be set to point it to the correct URL Scheme.

Environment:

fejta-bot commented 4 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot commented 4 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

fejta-bot commented 4 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /close

k8s-ci-robot commented 4 years ago

@fejta-bot: Closing this issue.

In response to [this](https://github.com/kubernetes-sigs/external-dns/issues/1457#issuecomment-667578063): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Send feedback to sig-testing, kubernetes/test-infra and/or [fejta](https://github.com/fejta). >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
vsabella commented 3 years ago

@flyer299 not sure if you fixed this - but setting "AZURE_ENVIRONMENT" as environment variable for pod