search

gravitational / teleport

The easiest, and most secure way to access and protect all of your infrastructure.
https://goteleport.com
GNU Affero General Public License v3.0
17.47k stars 1.75k forks source link

can't access application if acme domain use port other than 443 #22049

Closed dfang closed 1 month ago

dfang commented 1 year ago

Expected behavior: can access applications

Current behavior: access denied in web ui or nothing when access grafana.teleport.example.com:444

Bug details:

stevenGravy commented 1 year ago

@dfang this is a Let's Encrypt requirement to use 443 for challenges. You can install let's encrypt or other certificates directly if you want to use other prots.

dfang commented 1 year ago

@stevenGravy thanks for your reply, i can access WEB UI Portal by https://tele.example.com:444 (the certificate is issued), and i can connect to servers in web ui, i just can't access applications. what do you mean by install certificates directly ?

stevenGravy commented 1 year ago

@stevenGravy thanks for your reply, i can access WEB UI Portal by https://tele.example.com:444 (the certificate is issued), and i can connect to servers in web ui, i just can't access applications. what do you mean by install certificates directly ?

How are you managing TLS certs? Please share the configuration for the app access.

dfang commented 1 year ago

just change 443 to 444 in /etc/teleport.yaml

webvictim commented 1 year ago

I just reproduced this myself by running teleport configure with these arguments:

sudo teleport configure -o /etc/teleport.yaml \
  --acme --acme-email=acme@email.com \
  --cluster-name=gus-acme.example.com \
  --public-addr=gus-acme.example.com:3080

This gives me a default config file like this:

version: v3
teleport:
  nodename: ip-172-31-3-106
  data_dir: /var/lib/teleport
  log:
    output: stderr
    severity: INFO
    format:
      output: text
  ca_pin: ""
  diag_addr: ""
auth_service:
  enabled: "yes"
  listen_addr: 0.0.0.0:3025
  cluster_name: gus-acme.example.com
  proxy_listener_mode: multiplex
ssh_service:
  enabled: "yes"
  commands:
  - name: hostname
    command: [hostname]
    period: 1m0s
proxy_service:
  enabled: "yes"
  web_listen_addr: 0.0.0.0:3080
  public_addr: gus-acme.example.com:3080
  https_keypairs: []
  https_keypairs_reload_interval: 0s
  acme:
    enabled: "yes"
    email: acme@email.com

I also:

After this, Starting Teleport and hitting https://gus-acme.example.com:3080 in my browser results in incoming traffic from Let's Encrypt on port 443 (not 3080), then a failed challenge and an error being logged:

Aug 23 18:23:09 ip-172-31-3-106 kernel: [  384.520271] iptables: IN=eth0 OUT= MAC=02:3b:a1:8c:d3:ad:02:a7:4c:9a:67:2c:08:00 SRC=3.142.146.196 DST=172.31.3.106 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=17106 DF PROTO=TCP SPT=25732 DPT=443 WINDOW=62727 RES=0x00 SYN URGP=0
Aug 23 18:23:09 ip-172-31-3-106 kernel: [  384.580113] iptables: IN=eth0 OUT= MAC=02:3b:a1:8c:d3:ad:02:a7:4c:9a:67:2c:08:00 SRC=23.178.112.202 DST=172.31.3.106 LEN=60 TOS=0x00 PREC=0x00 TTL=46 ID=6444 DF PROTO=TCP SPT=19948 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Aug 23 18:23:10 ip-172-31-3-106 kernel: [  384.633040] iptables: IN=eth0 OUT= MAC=02:3b:a1:8c:d3:ad:02:a7:4c:9a:67:2c:08:00 SRC=35.90.20.57 DST=172.31.3.106 LEN=60 TOS=0x00 PREC=0x00 TTL=49 ID=54219 DF PROTO=TCP SPT=46464 DPT=443 WINDOW=62727 RES=0x00 SYN URGP=0

Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: 2023-08-23T18:23:11Z [ALPN:PROX] WARN Failed to handle client 
connection. error:[
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: ERROR REPORT:
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: Original Error: *errors.errorString acme/autocert: unable to satisfy "https://acme-v02.api.letsencrypt.org/acme/authz-v3/257682551106" for domain "gus-acme.example.com": no viable challenge type found
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: Stack Trace:
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: #011github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:392 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).handleConn
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: #011github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:326 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).Serve.func1
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: #011runtime/asm_amd64.s:1598 runtime.goexit
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: User Message: acme/autocert: unable to satisfy "https://acme-v02.api.letsencrypt.org/acme/authz-v3/257682551106" for domain "gus-acme.example.com": no viable challenge type found] alpnproxy/proxy.go:337
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: 2023-08-23T18:23:11Z [ALPN:PROX] WARN Failed to handle client connection. error:[
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: ERROR REPORT:
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: Original Error: *errors.errorString acme/autocert: missing certificate
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: Stack Trace:
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: #011github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:392 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).handleConn
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: #011github.com/gravitational/teleport/lib/srv/alpnproxy/proxy.go:326 github.com/gravitational/teleport/lib/srv/alpnproxy.(*Proxy).Serve.func1
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: #011runtime/asm_amd64.s:1598 runtime.goexit
Aug 23 18:23:11 ip-172-31-3-106 teleport[2007]: User Message: acme/autocert: missing certificate] alpnproxy/proxy.go:337

After a short delay, the browser shows an SSL error: Screenshot 2023-08-23 at 15 29 31

The root of the issue seems to be that the TLS-ALPN-01 challenge only works over port 443: https://letsencrypt.org/docs/challenge-types/#tls-alpn-01

Like TLS-SNI-01, it is performed via TLS on port 443.

I would suggest that the following list of circumstances ought to generate a fatal error where Teleport refuses to start:

Related issue: #29863

Still an issue in 13.3.5.

dfang commented 1 year ago

@webvictim since teleport can run behind nginx or caddy now, I don't need this feature.

webvictim commented 1 year ago

Indeed @dfang - but we should still fix it 😅

zmb3 commented 3 months ago

I would suggest that the following list of circumstances ought to generate a fatal error where Teleport refuses to start

Sounds good in theory, but I'm not sure it's worth the risk of breaking people. Feels like there are too many corner cases to handle.

webvictim commented 3 months ago

At this point anyone having the issue probably just googles it and finds this issue anyway...