search

gravitational / teleport

The easiest, and most secure way to access and protect all of your infrastructure.
https://goteleport.com
GNU Affero General Public License v3.0
17.47k stars 1.75k forks source link

Unable to access teleport https api with helm installation - "acme can't get a cert for domain" #6292

Closed dmitry-mightydevops closed 3 years ago

dmitry-mightydevops commented 3 years ago

Description

What happened:

Followed instructions here https://goteleport.com/docs/kubernetes-access/getting-started/

Kubernetes 1.19 running in AWS EKS with latest cert-manager and latest nginx-ingress controller.

After deploying with helm, I see the service created

✗  kgsvc                                          
NAME               TYPE           CLUSTER-IP      EXTERNAL-IP                                                              PORT(S)                                                      AGE
teleport-cluster   LoadBalancer   172.20.43.174   xxx-yyy.us-west-2.elb.amazonaws.com   443:30627/TCP,3023:32124/TCP,3026:31338/TCP,3024:32302/TCP   38m

curl https://teleport.team.com/webapi/ping
curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error

dns records added:

{
  "Comment": "Create records",
  "Changes": [
    {
      "Action": "CREATE",
      "ResourceRecordSet": {
        "Name": "teleport.team.com",
        "Type": "CNAME",
        "TTL": 300,
        "ResourceRecords": [
          {
            "Value": "xxx-yyy.us-west-2.elb.amazonaws.com"
          }
        ]
      }
    },
    {
      "Action": "CREATE",
      "ResourceRecordSet": {
        "Name": "*.teleport.team.com",
        "Type": "CNAME",
        "TTL": 300,
        "ResourceRecords": [
          {
            "Value": "xxx-yyy.us-west-2.elb.amazonaws.com"
          }
        ]
      }
    }
  ]
}

✗  klon teleport teleport-cluster-5556b67786-m4fj7 
INFO [PROC:1]    Service diag is creating new listener on 0.0.0.0:3000. service/signals.go:213
INFO [DIAG:1]    Starting diagnostic service on 0.0.0.0:3000. service/service.go:2049
INFO [PROC:1]    Service auth is creating new listener on 0.0.0.0:3025. service/signals.go:213
INFO [AUTH:1]    Starting Auth service with PROXY protocol support. service/service.go:1236
WARN [AUTH:1]    Configuration setting auth_service/advertise_ip is not set. guessing 10.110.8.4:3025. service/service.go:1314
[AUTH]         Auth service 6.0.2:v6.0.2-0-g1cb1420b7 is starting on 10.110.8.4:3025.
INFO [AUTH:1]    Auth service 6.0.2:v6.0.2-0-g1cb1420b7 is starting on 10.110.8.4:3025. utils/cli.go:226
INFO [PROC:1]    Connecting to the cluster team-com-eks with TLS client certificate. service/connect.go:128
INFO [PROC:1]    Connecting to the cluster team-com-eks with TLS client certificate. service/connect.go:128
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log. service/service.go:1895
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload. service/service.go:1895
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/sessions. service/service.go:1895
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/sessions/default. service/service.go:1895
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log. service/service.go:1895
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload. service/service.go:1895
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/streaming. service/service.go:1895
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/streaming/default. service/service.go:1895
INFO [PROC:1]    Service kubernetes is creating new listener on 0.0.0.0:3027. service/signals.go:213
INFO [KUBERNETE] Starting Kube service on [::]:3027. service/kubernetes.go:252
INFO [KUBERNETE] Kubernetes service 6.0.2:v6.0.2-0-g1cb1420b7 is starting on [::]:3027. utils/cli.go:226
INFO [PROC:1]    Service proxy:ssh is creating new listener on 0.0.0.0:3023. service/signals.go:213
INFO [PROC:1]    Service proxy:kube is creating new listener on 0.0.0.0:3026. service/signals.go:213
INFO [PROC:1]    Service proxy:web is creating new listener on 0.0.0.0:3080. service/signals.go:213
INFO [PROC:1]    Service proxy:tunnel is creating new listener on 0.0.0.0:3024. service/signals.go:213
[KUBERNETES]   Kubernetes service 6.0.2:v6.0.2-0-g1cb1420b7 is starting on [::]:3027.
INFO [PROXY:SER] Reverse tunnel service 6.0.2:v6.0.2-0-g1cb1420b7 is starting on 0.0.0.0:3024. utils/cli.go:226
[PROXY]        Reverse tunnel service 6.0.2:v6.0.2-0-g1cb1420b7 is starting on 0.0.0.0:3024.
INFO [PROXY:SER] Starting 6.0.2:v6.0.2-0-g1cb1420b7 on 0.0.0.0:3024 using sqlite cache that will expire after connection to database is lost after 20h0m0s, will cache frequently accessed items for 2s service/service.go:2460
INFO [PROXY:SER] Managing certs using ACME https://datatracker.ietf.org/doc/rfc8555/. service/service.go:2539
INFO [PROXY:SER] Web proxy service 6.0.2:v6.0.2-0-g1cb1420b7 is starting on 0.0.0.0:3080. utils/cli.go:226
[PROXY]        Web proxy service 6.0.2:v6.0.2-0-g1cb1420b7 is starting on 0.0.0.0:3080.
INFO [PROXY:SER] Web proxy service 6.0.2:v6.0.2-0-g1cb1420b7 is starting on 0.0.0.0:3080. service/service.go:2587
INFO [PROXY:SER] SSH proxy service 6.0.2:v6.0.2-0-g1cb1420b7 is starting on [::]:3023. utils/cli.go:226
[PROXY]        SSH proxy service 6.0.2:v6.0.2-0-g1cb1420b7 is starting on [::]:3023.
INFO [PROXY:SER] SSH proxy service 6.0.2:v6.0.2-0-g1cb1420b7 is starting on {[::]:3023 tcp } service/service.go:2632
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log. service/service.go:1895
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload. service/service.go:1895
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/sessions. service/service.go:1895
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/sessions/default. service/service.go:1895
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log. service/service.go:1895
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload. service/service.go:1895
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/streaming. service/service.go:1895
INFO [AUDIT:1]   Creating directory /var/lib/teleport/log/upload/streaming/default. service/service.go:1895
INFO [DB:SERVIC] Starting Database proxy server on 0.0.0.0:3080. service/service.go:2750
ERRO [PROXY:SER] "proxy2021/04/02 01:05:09 http: TLS handshake error from 10.110.2.100:33666: acme can't get a cert for domain teleport.team.com, add it to the proxy_service.public_addr, or use one of the domains: team-cluster-eks\n" utils/cli.go:287
ERRO [PROXY:SER] "proxy2021/04/02 01:05:11 http: TLS handshake error from 10.110.3.7:63990: acme can't get a cert for domain teleport.team.com, add it to the proxy_service.public_addr, or use one of the domains: team-cluster-eks\n" utils/cli.go:287

What you expected to happen:

Teleport accessible via curl at least.

Server Details

dmitry-mightydevops commented 3 years ago

Ok the issue was with the values of the helm chart

clusterName: "team-cluster-eks"
acme: true
acmeEmail: devops@saritasa.com
teleportVersionOverride: 6.0.2

So I thought that cluster name is the one we use in AWS EKS, turns out that should be a FQDN, i.e. teleport.team.com.

I would find the name of the variable confusing and would recommend using something like "clusterUrl" or "clusterDomain", as it's used here

webvictim commented 3 years ago

The Helm docs and README are being rewritten as part of #6390 and #6344 respectively - it's very clear that clusterName must be an FQDN now.