search

kubernetes-sigs / external-dns

Configure external DNS servers (AWS Route53, Google CloudDNS and others) for Kubernetes Ingresses and Services
Apache License 2.0
7.73k stars 2.57k forks source link

Can't get AWS how-to to run: No endpoints could be generated from service default/nginx #889

Closed dirkgomez closed 5 years ago

dirkgomez commented 5 years ago

I'm trying to get external-dns to work in AWS with the provided Nginx example (https://github.com/kubernetes-incubator/external-dns/blob/master/docs/tutorials/aws.md)

Here's my logfile snippet:

time="2019-02-07T08:36:36Z" level=info msg="config: {Master: KubeConfig: RequestTimeout:30s IstioIngressGateway:istio-system/istio-ingressgateway Sources:[service ingress] Namespace: AnnotationFilter: FQDNTemplate: CombineFQDNAndAnnotation:false Compatibility: PublishInternal:false PublishHostIP:false ConnectorSourceServer:localhost:8080 Provider:aws GoogleProject: DomainFilter:[redacted.dirkgomez.de] ZoneIDFilter:[] AlibabaCloudConfigFile:/etc/kubernetes/alibaba-cloud.json AlibabaCloudZoneType: AWSZoneType:private AWSZoneTagFilter:[] AWSAssumeRole: AWSBatchChangeSize:1000 AWSBatchChangeInterval:1s AWSEvaluateTargetHealth:true AWSAPIRetries:3 AzureConfigFile:/etc/kubernetes/azure.json AzureResourceGroup: CloudflareProxied:false CloudflareZonesPerPage:50 InfobloxGridHost: InfobloxWapiPort:443 InfobloxWapiUsername:admin InfobloxWapiPassword: InfobloxWapiVersion:2.3.1 InfobloxSSLVerify:true DynCustomerName: DynUsername: DynPassword: DynMinTTLSeconds:0 OCIConfigFile:/etc/kubernetes/oci.yaml InMemoryZones:[] PDNSServer:http://localhost:8081 PDNSAPIKey: PDNSTLSEnabled:false TLSCA: TLSClientCert: TLSClientCertKey: Policy:upsert-only Registry:txt TXTOwnerID:/hostedzone/REDACTED TXTPrefix: Interval:1m0s Once:false DryRun:false LogFormat:text MetricsAddress::7979 LogLevel:debug TXTCacheInterval:0s ExoscaleEndpoint:https://api.exoscale.ch/dns ExoscaleAPIKey: ExoscaleAPISecret: CRDSourceAPIVersion:externaldns.k8s.io/v1alpha1 CRDSourceKind:DNSEndpoint ServiceTypeFilter:[] RFC2136Host: RFC2136Port:0 RFC2136Zone: RFC2136Insecure:false RFC2136TSIGKeyName: RFC2136TSIGSecret: RFC2136TSIGSecretAlg: RFC2136TAXFR:false}"
time="2019-02-07T08:36:36Z" level=info msg="Created Kubernetes client https://172.20.0.1:443"
time="2019-02-07T08:36:36Z" level=debug msg="Considering zone: /hostedzone/REDACTED (domain: redacted.dirkgomez.de.)"
time="2019-02-07T08:36:36Z" level=debug msg="No endpoints could be generated from service default/kubernetes"
time="2019-02-07T08:36:36Z" level=debug msg="No endpoints could be generated from service default/nginx"
time="2019-02-07T08:36:36Z" level=debug msg="No endpoints could be generated from service kube-system/dashboard-kubernetes-dashboard"
time="2019-02-07T08:36:36Z" level=debug msg="No endpoints could be generated from service kube-system/kube-dns"
time="2019-02-07T08:36:36Z" level=debug msg="No endpoints could be generated from service kube-system/tiller-deploy"

The AWS zone type is private: --aws-zone-type=private

Where does it get the 172.20.0.1 from? And any pointers why endpoints cannot be generated?

Evesy commented 5 years ago

Hey @dirkgomez

172.20.0.1 will be the address of the Kubernetes API service (e.g. kubectl get service -n default kubernetes).

Those services it mentions being unable to generate endpoints for all look like they're probably ClusterIP services (perhaps the exception being Nginx?). For ClusterIP services the default behaviour is to not create DNS records for those. If you do want to create DNS for those you can add the flag --publish-internal to your deployment

ghost commented 5 years ago

That flag should be:

--publish-internal-services

fejta-bot commented 5 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

drewhemm commented 5 years ago

I have encountered the same symptom:

$ external-dns --registry txt --txt-owner-id "REDACTED" --provider aws --aws-zone-type=private --policy=upsert-only --domain-filter=e2enft.us-west-2.dev-aws.REDACTED.com --source=ingress --source=service --log-level=debug --dry-run --metrics-address=":7980" --publish-internal-services
INFO[0000] config: {Master: KubeConfig: RequestTimeout:30s IstioIngressGatewayServices:[istio-system/istio-ingressgateway] ContourLoadBalancerService:heptio-contour/contour Sources:[ingress service] Namespace: AnnotationFilter: FQDNTemplate: CombineFQDNAndAnnotation:false IgnoreHostnameAnnotation:false Compatibility: PublishInternal:true PublishHostIP:false ConnectorSourceServer:localhost:8080 Provider:aws GoogleProject: DomainFilter:[e2enft.us-west-2.dev-aws.REDACTED.com] ExcludeDomains:[] ZoneIDFilter:[] AlibabaCloudConfigFile:/etc/kubernetes/alibaba-cloud.json AlibabaCloudZoneType: AWSZoneType:private AWSZoneTagFilter:[] AWSAssumeRole: AWSBatchChangeSize:1000 AWSBatchChangeInterval:1s AWSEvaluateTargetHealth:true AWSAPIRetries:3 AWSPreferCNAME:false AzureConfigFile:/etc/kubernetes/azure.json AzureResourceGroup: CloudflareProxied:false CloudflareZonesPerPage:50 CoreDNSPrefix:/skydns/ RcodezeroTXTEncrypt:false InfobloxGridHost: InfobloxWapiPort:443 InfobloxWapiUsername:admin InfobloxWapiPassword: InfobloxWapiVersion:2.3.1 InfobloxSSLVerify:true InfobloxView: InfobloxMaxResults:0 DynCustomerName: DynUsername: DynPassword: DynMinTTLSeconds:0 OCIConfigFile:/etc/kubernetes/oci.yaml InMemoryZones:[] PDNSServer:http://localhost:8081 PDNSAPIKey: PDNSTLSEnabled:false TLSCA: TLSClientCert: TLSClientCertKey: Policy:upsert-only Registry:txt TXTOwnerID:REDACTED TXTPrefix: Interval:1m0s Once:false DryRun:true LogFormat:text MetricsAddress::7980 LogLevel:debug TXTCacheInterval:0s ExoscaleEndpoint:https://api.exoscale.ch/dns ExoscaleAPIKey: ExoscaleAPISecret: CRDSourceAPIVersion:externaldns.k8s.io/v1alpha1 CRDSourceKind:DNSEndpoint ServiceTypeFilter:[] CFAPIEndpoint: CFUsername: CFPassword: RFC2136Host: RFC2136Port:0 RFC2136Zone: RFC2136Insecure:false RFC2136TSIGKeyName: RFC2136TSIGSecret: RFC2136TSIGSecretAlg: RFC2136TAXFR:false NS1Endpoint: NS1IgnoreSSL:false TransIPAccountName: TransIPPrivateKeyFile:} 
INFO[0000] running in dry-run mode. No changes to DNS records will be made. 
INFO[0000] Created Kubernetes client https://10.100.0.1:443 
DEBU[0001] node added                                   
... (lots of these lines)                                         
DEBU[0001] node added                                   
DEBU[0001] service added                                
... (lots of these lines)         
DEBU[0001] service added                                
DEBU[0001] pod added                                    
... (lots of these lines)                                 
DEBU[0001] pod added                                    
DEBU[0003] Considering zone: /hostedzone/REDACTED (domain: e2enft.us-west-2.dev-aws.REDACTED.com.) 
DEBU[0003] No endpoints could be generated from ingress redis/stunnel 
DEBU[0003] No endpoints could be generated from ingress monitoring/alertmanager 
DEBU[0003] No endpoints could be generated from ingress monitoring/grafana 
DEBU[0003] No endpoints could be generated from ingress monitoring/prometheus 
DEBU[0003] No endpoints could be generated from service monitoring/custom-metrics-apiserver 
DEBU[0003] No endpoints could be generated from service monitoring/grafana 
DEBU[0003] No endpoints could be generated from service monitoring/kube-state-metrics 
DEBU[0003] No endpoints could be generated from service kube-system/kube-dns 
DEBU[0003] No endpoints could be generated from service redis/stunnel 
DEBU[0003] No endpoints could be generated from service monitoring/prometheus 
DEBU[0003] No endpoints could be generated from service ci/jenkins-master 
DEBU[0003] No endpoints could be generated from service default/kubernetes 
DEBU[0003] No endpoints could be generated from service ingress-nginx/ingress-nginx 
DEBU[0003] No endpoints could be generated from service monitoring/alertmanager 
DEBU[0003] No endpoints could be generated from service monitoring/grafana-rds 
DEBU[0003] No endpoints could be generated from service monitoring/grafana-mysql 
DEBU[0003] Considering zone: /hostedzone/REDACTED (domain: e2enft.us-west-2.dev-aws.REDACTED.com.) 
INFO[0003] All records are already up to date

We have the same setup working in another AWS account with a public Route53 hosted zone (instead of private here), other than that, I do not believe there are any differences. In this account, no records are added to the hosted zone, but also no errors are thrown. Removing --dry-run makes no difference.

We are running v0.5.16

drewhemm commented 5 years ago

Ah, in this case, it's because the load balancer doesn't yet exist, which is a problem not related to external-dns

fejta-bot commented 5 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

fejta-bot commented 5 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /close

k8s-ci-robot commented 5 years ago

@fejta-bot: Closing this issue.

In response to [this](https://github.com/kubernetes-incubator/external-dns/issues/889#issuecomment-544998571): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Send feedback to sig-testing, kubernetes/test-infra and/or [fejta](https://github.com/fejta). >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.