search

kubernetes / ingress-nginx

Ingress-NGINX Controller for Kubernetes
https://kubernetes.github.io/ingress-nginx/
Apache License 2.0
16.96k stars 8.14k forks source link

server-alias pcre regex uses the ingress-nginx default tls certificate but redirect to the correct backend which has a valid * certificate that match #10953

Open Julien-Beezeelinx opened 5 months ago

Julien-Beezeelinx commented 5 months ago

What happened: Using regex in server alias of an ingress, but ingress-nginx uses it's default tls certificate instead of the one of the ingress.

What you expected to happen: I expect ingress-nginx to use the available valid certificate for the regex I defined in the server-alias annotations and not it's default certificate.

What do you think went wrong?: There must be some kind of cache in the TLS certificate inclusion/matching in ingress-nginx that is updated using the defined ingress rules but not using the server-alias annotation. Because when I simply use a regex in server-alias it does not work. But if I add an host with the same *.subdomain.domain.com as the certificate in the ingress (and matching the regex in server-alias), and then remove this host. That server alias uses the correct tls certificate until ingress-nginx restart. What is really strange to me is even if ingress-nginx uses the wrong certificate, it forwards the request to the correct backend....

NGINX Ingress controller version (exec into the pod and run nginx-ingress-controller --version.):

 /nginx-ingress-controller --version
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v1.9.6
  Build:         6a73aa3b05040a97ef8213675a16142a9c95952a
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.21.6

-------------------------------------------------------------------------------

Kubernetes version (use kubectl version): v1.25.9

Environment:

How to reproduce this issue:

Install an application that will act as default backend (is just an echo app)

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/docs/examples/http-svc.yaml

Create an ingress (please add any additional annotation required)

Need tls certificate for this ingress (example using cert-manager)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: foo
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.ingress.kubernetes.io/server-alias: '~^(?!test).+\.subdomain\.domain\.com$'
spec:
  tls:
    - hosts:
      - '*.subdomain.domain.com'
      - 'subdomain.domain.com'
      secretName: tls-secret
  rules:
    - host: 'subdomain.domain.com'
      http:
        paths:
          - pathType: ImplementationSpecific
            backend:
              service:
                name: http-svc
                port:
                  number: 80
  ingressClassName: nginx
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: bar
  annotations:
    nginx.ingress.kubernetes.io/server-alias: '~^test-(.+)\.subdomain\.domain\.com$'
spec:
  tls:
    - hosts:
      - '*.subdomain.domain.com'
      - 'subdomain.domain.com'
      secretName: tls-secret
  rules:
    - host: 'test.subdomain.domain.com'
      http:
        paths:
          - pathType: ImplementationSpecific
            backend:
              service:
                name: http-svc
                port:
                  number: 80
  ingressClassName: nginx
Relevant server block in the nginx.conf ``` ## start server subdomain.domain.com server { server_name subdomain.domain.com ~^(?!test).+\.subdomain\.domain\.com$ ; listen 80 proxy_protocol ; listen [::]:80 proxy_protocol ; listen 443 proxy_protocol ssl ; listen [::]:443 proxy_protocol ssl ; set $proxy_upstream_name "-"; ssl_certificate_by_lua_block { certificate.call() } location / { set $namespace "test1"; set $ingress_name "foo"; set $service_name "echo1"; set $service_port "80"; set $location_path "/"; set $global_rate_limit_exceeding n; rewrite_by_lua_block { lua_ingress.rewrite({ force_ssl_redirect = false, ssl_redirect = true, force_no_ssl_redirect = false, preserve_trailing_slash = false, use_port_in_redirects = false, global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } }, }) balancer.rewrite() plugins.run() } # be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any # will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)` # other authentication method such as basic auth or external auth useless - all requests will be allowed. #access_by_lua_block { #} header_filter_by_lua_block { lua_ingress.header() plugins.run() } body_filter_by_lua_block { plugins.run() } log_by_lua_block { balancer.log() monitor.call() plugins.run() } port_in_redirect off; set $balancer_ewma_score -1; set $proxy_upstream_name "test1-echo1-80"; set $proxy_host $proxy_upstream_name; set $pass_access_scheme $scheme; set $pass_server_port $proxy_protocol_server_port; set $best_http_host $http_host; set $pass_port $pass_server_port; set $proxy_alternative_upstream_name ""; client_max_body_size 1m; proxy_set_header Host $best_http_host; # Pass the extracted client certificate to the backend # Allow websocket connections proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_set_header X-Request-ID $req_id; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $best_http_host; proxy_set_header X-Forwarded-Port $pass_port; proxy_set_header X-Forwarded-Proto $pass_access_scheme; proxy_set_header X-Forwarded-Scheme $pass_access_scheme; proxy_set_header X-Scheme $pass_access_scheme; # Pass the original X-Forwarded-For proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for; # mitigate HTTPoxy Vulnerability # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/ proxy_set_header Proxy ""; # Custom headers to proxied server proxy_connect_timeout 300s; proxy_send_timeout 300s; proxy_read_timeout 300s; proxy_buffering off; proxy_buffer_size 4k; proxy_buffers 4 4k; proxy_max_temp_file_size 1024m; proxy_request_buffering on; proxy_http_version 1.1; proxy_cookie_domain off; proxy_cookie_path off; # In case of errors try the next upstream server before returning an error proxy_next_upstream error timeout; proxy_next_upstream_timeout 0; proxy_next_upstream_tries 3; # Custom code snippet configured in the configuration configmap proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://upstream_balancer; proxy_redirect off; } # Custom code snippet configured in the configuration configmap # Prevent clickjacking attacks # Comment out this line if you want to make use of iframes add_header X-Frame-Options SAMEORIGIN; # Prevent "mime" based attacks add_header X-Content-Type-Options nosniff; # Prevent XSS attacks add_header X-XSS-Protection "1; mode=block"; } ## end server subdomain.domain.com ## start server test.subdomain.domain.com server { server_name test.subdomain.domain.com ~^test-(.+)\.subdomain\.domain\.com$ ; listen 80 proxy_protocol ; listen [::]:80 proxy_protocol ; listen 443 proxy_protocol ssl ; listen [::]:443 proxy_protocol ssl ; set $proxy_upstream_name "-"; ssl_certificate_by_lua_block { certificate.call() } # Custom code snippet configured for host test.subdomain.domain.com ssl_verify_client optional_no_ca; location / { set $namespace "test1"; set $ingress_name "bar"; set $service_name "echo2"; set $service_port "80"; set $location_path "/"; set $global_rate_limit_exceeding n; rewrite_by_lua_block { lua_ingress.rewrite({ force_ssl_redirect = false, ssl_redirect = true, force_no_ssl_redirect = false, preserve_trailing_slash = false, use_port_in_redirects = false, global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } }, }) balancer.rewrite() plugins.run() } # be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any # will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)` # other authentication method such as basic auth or external auth useless - all requests will be allowed. #access_by_lua_block { #} header_filter_by_lua_block { lua_ingress.header() plugins.run() } body_filter_by_lua_block { plugins.run() } log_by_lua_block { balancer.log() monitor.call() plugins.run() } port_in_redirect off; set $balancer_ewma_score -1; set $proxy_upstream_name "test1-echo2-80"; set $proxy_host $proxy_upstream_name; set $pass_access_scheme $scheme; set $pass_server_port $proxy_protocol_server_port; set $best_http_host $http_host; set $pass_port $pass_server_port; set $proxy_alternative_upstream_name ""; client_max_body_size 1m; proxy_set_header Host $best_http_host; # Pass the extracted client certificate to the backend # Allow websocket connections proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_set_header X-Request-ID $req_id; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $best_http_host; proxy_set_header X-Forwarded-Port $pass_port; proxy_set_header X-Forwarded-Proto $pass_access_scheme; proxy_set_header X-Forwarded-Scheme $pass_access_scheme; proxy_set_header X-Scheme $pass_access_scheme; # Pass the original X-Forwarded-For proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for; # mitigate HTTPoxy Vulnerability # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/ proxy_set_header Proxy ""; # Custom headers to proxied server proxy_connect_timeout 300s; proxy_send_timeout 300s; proxy_read_timeout 300s; proxy_buffering off; proxy_buffer_size 4k; proxy_buffers 4 4k; proxy_max_temp_file_size 1024m; proxy_request_buffering on; proxy_http_version 1.1; proxy_cookie_domain off; proxy_cookie_path off; # In case of errors try the next upstream server before returning an error proxy_next_upstream error timeout; proxy_next_upstream_timeout 0; proxy_next_upstream_tries 3; set $svr_ssl_client_cert ''; access_by_lua_block { ngx.var.svr_ssl_client_cert = ngx.var.ssl_client_escaped_cert or ngx.req.get_headers()["X-SSL-Client-Cert"] or "" } proxy_set_header X-SSL-Client-Cert $svr_ssl_client_cert; # Custom code snippet configured in the configuration configmap proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://upstream_balancer; proxy_redirect off; } # Custom code snippet configured in the configuration configmap # Prevent clickjacking attacks # Comment out this line if you want to make use of iframes add_header X-Frame-Options SAMEORIGIN; # Prevent "mime" based attacks add_header X-Content-Type-Options nosniff; # Prevent XSS attacks add_header X-XSS-Protection "1; mode=block"; } ## end server test.subdomain.domain.com ```

make a request

# Uses correct certificate
curl -v subdomain.domain.com
curl -v test.subdomain.domain.com

# Uses wrong certificate
curl -v random.subdomain.domain.com
curl -v test-random.subdomain.domain.com

Anything else we need to know: if you apply the following ingress definition, and then re-apply the first one. The correct certficate match with the server-alias regex until ingress-nginx restart and start using it's default certificate all over again...

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: foo
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.ingress.kubernetes.io/server-alias: '~^(?!test).+\.subdomain\.domain\.com$'
spec:
  tls:
    - hosts:
      - '*.subdomain.domain.com'
      - 'subdomain.domain.com'
      secretName: tls-secret
  rules:
    - host: 'subdomain.domain.com'
      http: &paths
        paths:
          - pathType: ImplementationSpecific
            backend:
              service:
                name: http-svc
                port:
                  number: 80
    - host: '*.subdomain.domain.com'
      http: *paths
  ingressClassName: nginx
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: bar
  annotations:
    nginx.ingress.kubernetes.io/server-alias: '~^test-(.+)\.subdomain\.domain\.com$'
spec:
  tls:
    - hosts:
      - '*.subdomain.domain.com'
      - 'subdomain.domain.com'
      secretName: tls-secret
  rules:
    - host: 'test.subdomain.domain.com'
      http:
        paths:
          - pathType: ImplementationSpecific
            backend:
              service:
                name: http-svc
                port:
                  number: 80
  ingressClassName: nginx
Relevant server block in the nginx.conf ``` ## start server *.subdomain.domain.com server { server_name ~^(?[\w-]+)\.subdomain\.domain\.com$ ~^(?!test).+\.subdomain\.domain\.com$ ; listen 80 proxy_protocol ; listen [::]:80 proxy_protocol ; listen 443 proxy_protocol ssl ; listen [::]:443 proxy_protocol ssl ; set $proxy_upstream_name "-"; ssl_certificate_by_lua_block { certificate.call() } location / { set $namespace "test1"; set $ingress_name "foo"; set $service_name "echo1"; set $service_port "80"; set $location_path "/"; set $global_rate_limit_exceeding n; rewrite_by_lua_block { lua_ingress.rewrite({ force_ssl_redirect = false, ssl_redirect = true, force_no_ssl_redirect = false, preserve_trailing_slash = false, use_port_in_redirects = false, global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } }, }) balancer.rewrite() plugins.run() } # be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any # will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)` # other authentication method such as basic auth or external auth useless - all requests will be allowed. #access_by_lua_block { #} header_filter_by_lua_block { lua_ingress.header() plugins.run() } body_filter_by_lua_block { plugins.run() } log_by_lua_block { balancer.log() monitor.call() plugins.run() } port_in_redirect off; set $balancer_ewma_score -1; set $proxy_upstream_name "test1-echo1-80"; set $proxy_host $proxy_upstream_name; set $pass_access_scheme $scheme; set $pass_server_port $proxy_protocol_server_port; set $best_http_host $http_host; set $pass_port $pass_server_port; set $proxy_alternative_upstream_name ""; client_max_body_size 1m; proxy_set_header Host $best_http_host; # Pass the extracted client certificate to the backend # Allow websocket connections proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_set_header X-Request-ID $req_id; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $best_http_host; proxy_set_header X-Forwarded-Port $pass_port; proxy_set_header X-Forwarded-Proto $pass_access_scheme; proxy_set_header X-Forwarded-Scheme $pass_access_scheme; proxy_set_header X-Scheme $pass_access_scheme; # Pass the original X-Forwarded-For proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for; # mitigate HTTPoxy Vulnerability # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/ proxy_set_header Proxy ""; # Custom headers to proxied server proxy_connect_timeout 300s; proxy_send_timeout 300s; proxy_read_timeout 300s; proxy_buffering off; proxy_buffer_size 4k; proxy_buffers 4 4k; proxy_max_temp_file_size 1024m; proxy_request_buffering on; proxy_http_version 1.1; proxy_cookie_domain off; proxy_cookie_path off; # In case of errors try the next upstream server before returning an error proxy_next_upstream error timeout; proxy_next_upstream_timeout 0; proxy_next_upstream_tries 3; # Custom code snippet configured in the configuration configmap proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://upstream_balancer; proxy_redirect off; } # Custom code snippet configured in the configuration configmap # Prevent clickjacking attacks # Comment out this line if you want to make use of iframes add_header X-Frame-Options SAMEORIGIN; # Prevent "mime" based attacks add_header X-Content-Type-Options nosniff; # Prevent XSS attacks add_header X-XSS-Protection "1; mode=block"; } ## end server *.subdomain.domain.com ## start server subdomain.domain.com server { server_name subdomain.domain.com ~^(?!test).+\.subdomain\.domain\.com$ ; listen 80 proxy_protocol ; listen [::]:80 proxy_protocol ; listen 443 proxy_protocol ssl ; listen [::]:443 proxy_protocol ssl ; set $proxy_upstream_name "-"; ssl_certificate_by_lua_block { certificate.call() } location / { set $namespace "test1"; set $ingress_name "foo"; set $service_name "echo1"; set $service_port "80"; set $location_path "/"; set $global_rate_limit_exceeding n; rewrite_by_lua_block { lua_ingress.rewrite({ force_ssl_redirect = false, ssl_redirect = true, force_no_ssl_redirect = false, preserve_trailing_slash = false, use_port_in_redirects = false, global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } }, }) balancer.rewrite() plugins.run() } # be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any # will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)` # other authentication method such as basic auth or external auth useless - all requests will be allowed. #access_by_lua_block { #} header_filter_by_lua_block { lua_ingress.header() plugins.run() } body_filter_by_lua_block { plugins.run() } log_by_lua_block { balancer.log() monitor.call() plugins.run() } port_in_redirect off; set $balancer_ewma_score -1; set $proxy_upstream_name "test1-echo1-80"; set $proxy_host $proxy_upstream_name; set $pass_access_scheme $scheme; set $pass_server_port $proxy_protocol_server_port; set $best_http_host $http_host; set $pass_port $pass_server_port; set $proxy_alternative_upstream_name ""; client_max_body_size 1m; proxy_set_header Host $best_http_host; # Pass the extracted client certificate to the backend # Allow websocket connections proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_set_header X-Request-ID $req_id; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $best_http_host; proxy_set_header X-Forwarded-Port $pass_port; proxy_set_header X-Forwarded-Proto $pass_access_scheme; proxy_set_header X-Forwarded-Scheme $pass_access_scheme; proxy_set_header X-Scheme $pass_access_scheme; # Pass the original X-Forwarded-For proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for; # mitigate HTTPoxy Vulnerability # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/ proxy_set_header Proxy ""; # Custom headers to proxied server proxy_connect_timeout 300s; proxy_send_timeout 300s; proxy_read_timeout 300s; proxy_buffering off; proxy_buffer_size 4k; proxy_buffers 4 4k; proxy_max_temp_file_size 1024m; proxy_request_buffering on; proxy_http_version 1.1; proxy_cookie_domain off; proxy_cookie_path off; # In case of errors try the next upstream server before returning an error proxy_next_upstream error timeout; proxy_next_upstream_timeout 0; proxy_next_upstream_tries 3; # Custom code snippet configured in the configuration configmap proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://upstream_balancer; proxy_redirect off; } # Custom code snippet configured in the configuration configmap # Prevent clickjacking attacks # Comment out this line if you want to make use of iframes add_header X-Frame-Options SAMEORIGIN; # Prevent "mime" based attacks add_header X-Content-Type-Options nosniff; # Prevent XSS attacks add_header X-XSS-Protection "1; mode=block"; } ## end server subdomain.domain.com ## start server test.subdomain.domain.com server { server_name test.subdomain.domain.com ~^test-(.+)\.subdomain\.domain\.com$ ; listen 80 proxy_protocol ; listen [::]:80 proxy_protocol ; listen 443 proxy_protocol ssl ; listen [::]:443 proxy_protocol ssl ; set $proxy_upstream_name "-"; ssl_certificate_by_lua_block { certificate.call() } # Custom code snippet configured for host test.subdomain.domain.com ssl_verify_client optional_no_ca; location / { set $namespace "test1"; set $ingress_name "bar"; set $service_name "echo2"; set $service_port "80"; set $location_path "/"; set $global_rate_limit_exceeding n; rewrite_by_lua_block { lua_ingress.rewrite({ force_ssl_redirect = false, ssl_redirect = true, force_no_ssl_redirect = false, preserve_trailing_slash = false, use_port_in_redirects = false, global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } }, }) balancer.rewrite() plugins.run() } # be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any # will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)` # other authentication method such as basic auth or external auth useless - all requests will be allowed. #access_by_lua_block { #} header_filter_by_lua_block { lua_ingress.header() plugins.run() } body_filter_by_lua_block { plugins.run() } log_by_lua_block { balancer.log() monitor.call() plugins.run() } port_in_redirect off; set $balancer_ewma_score -1; set $proxy_upstream_name "test1-echo2-80"; set $proxy_host $proxy_upstream_name; set $pass_access_scheme $scheme; set $pass_server_port $proxy_protocol_server_port; set $best_http_host $http_host; set $pass_port $pass_server_port; set $proxy_alternative_upstream_name ""; client_max_body_size 1m; proxy_set_header Host $best_http_host; # Pass the extracted client certificate to the backend # Allow websocket connections proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_set_header X-Request-ID $req_id; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $best_http_host; proxy_set_header X-Forwarded-Port $pass_port; proxy_set_header X-Forwarded-Proto $pass_access_scheme; proxy_set_header X-Forwarded-Scheme $pass_access_scheme; proxy_set_header X-Scheme $pass_access_scheme; # Pass the original X-Forwarded-For proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for; # mitigate HTTPoxy Vulnerability # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/ proxy_set_header Proxy ""; # Custom headers to proxied server proxy_connect_timeout 300s; proxy_send_timeout 300s; proxy_read_timeout 300s; proxy_buffering off; proxy_buffer_size 4k; proxy_buffers 4 4k; proxy_max_temp_file_size 1024m; proxy_request_buffering on; proxy_http_version 1.1; proxy_cookie_domain off; proxy_cookie_path off; # In case of errors try the next upstream server before returning an error proxy_next_upstream error timeout; proxy_next_upstream_timeout 0; proxy_next_upstream_tries 3; set $svr_ssl_client_cert ''; access_by_lua_block { ngx.var.svr_ssl_client_cert = ngx.var.ssl_client_escaped_cert or ngx.req.get_headers()["X-SSL-Client-Cert"] or "" } proxy_set_header X-SSL-Client-Cert $svr_ssl_client_cert; # Custom code snippet configured in the configuration configmap proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://upstream_balancer; proxy_redirect off; } # Custom code snippet configured in the configuration configmap # Prevent clickjacking attacks # Comment out this line if you want to make use of iframes add_header X-Frame-Options SAMEORIGIN; # Prevent "mime" based attacks add_header X-Content-Type-Options nosniff; # Prevent XSS attacks add_header X-XSS-Protection "1; mode=block"; } ## end server test.subdomain.domain.com ```
Relevant server block in the nginx.conf if you then re-apply the first ingress to have the expected behaviour of the regex rules in the server-alias annotations, ``` ## start server subdomain.domain.com server { server_name subdomain.domain.com ~^(?!test).+\.subdomain\.domain\.com$ ; listen 80 proxy_protocol ; listen [::]:80 proxy_protocol ; listen 443 proxy_protocol ssl ; listen [::]:443 proxy_protocol ssl ; set $proxy_upstream_name "-"; ssl_certificate_by_lua_block { certificate.call() } location / { set $namespace "test1"; set $ingress_name "foo"; set $service_name "echo1"; set $service_port "80"; set $location_path "/"; set $global_rate_limit_exceeding n; rewrite_by_lua_block { lua_ingress.rewrite({ force_ssl_redirect = false, ssl_redirect = true, force_no_ssl_redirect = false, preserve_trailing_slash = false, use_port_in_redirects = false, global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } }, }) balancer.rewrite() plugins.run() } # be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any # will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)` # other authentication method such as basic auth or external auth useless - all requests will be allowed. #access_by_lua_block { #} header_filter_by_lua_block { lua_ingress.header() plugins.run() } body_filter_by_lua_block { plugins.run() } log_by_lua_block { balancer.log() monitor.call() plugins.run() } port_in_redirect off; set $balancer_ewma_score -1; set $proxy_upstream_name "test1-echo1-80"; set $proxy_host $proxy_upstream_name; set $pass_access_scheme $scheme; set $pass_server_port $proxy_protocol_server_port; set $best_http_host $http_host; set $pass_port $pass_server_port; set $proxy_alternative_upstream_name ""; client_max_body_size 1m; proxy_set_header Host $best_http_host; # Pass the extracted client certificate to the backend # Allow websocket connections proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_set_header X-Request-ID $req_id; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $best_http_host; proxy_set_header X-Forwarded-Port $pass_port; proxy_set_header X-Forwarded-Proto $pass_access_scheme; proxy_set_header X-Forwarded-Scheme $pass_access_scheme; proxy_set_header X-Scheme $pass_access_scheme; # Pass the original X-Forwarded-For proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for; # mitigate HTTPoxy Vulnerability # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/ proxy_set_header Proxy ""; # Custom headers to proxied server proxy_connect_timeout 300s; proxy_send_timeout 300s; proxy_read_timeout 300s; proxy_buffering off; proxy_buffer_size 4k; proxy_buffers 4 4k; proxy_max_temp_file_size 1024m; proxy_request_buffering on; proxy_http_version 1.1; proxy_cookie_domain off; proxy_cookie_path off; # In case of errors try the next upstream server before returning an error proxy_next_upstream error timeout; proxy_next_upstream_timeout 0; proxy_next_upstream_tries 3; # Custom code snippet configured in the configuration configmap proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://upstream_balancer; proxy_redirect off; } # Custom code snippet configured in the configuration configmap # Prevent clickjacking attacks # Comment out this line if you want to make use of iframes add_header X-Frame-Options SAMEORIGIN; # Prevent "mime" based attacks add_header X-Content-Type-Options nosniff; # Prevent XSS attacks add_header X-XSS-Protection "1; mode=block"; } ## end server subdomain.domain.com ## start server test.subdomain.domain.com server { server_name test.subdomain.domain.com ~^test-(.+)\.subdomain\.domain\.com$ ; listen 80 proxy_protocol ; listen [::]:80 proxy_protocol ; listen 443 proxy_protocol ssl ; listen [::]:443 proxy_protocol ssl ; set $proxy_upstream_name "-"; ssl_certificate_by_lua_block { certificate.call() } # Custom code snippet configured for host test.subdomain.domain.com ssl_verify_client optional_no_ca; location / { set $namespace "test1"; set $ingress_name "bar"; set $service_name "echo2"; set $service_port "80"; set $location_path "/"; set $global_rate_limit_exceeding n; rewrite_by_lua_block { lua_ingress.rewrite({ force_ssl_redirect = false, ssl_redirect = true, force_no_ssl_redirect = false, preserve_trailing_slash = false, use_port_in_redirects = false, global_throttle = { namespace = "", limit = 0, window_size = 0, key = { }, ignored_cidrs = { } }, }) balancer.rewrite() plugins.run() } # be careful with `access_by_lua_block` and `satisfy any` directives as satisfy any # will always succeed when there's `access_by_lua_block` that does not have any lua code doing `ngx.exit(ngx.DECLINED)` # other authentication method such as basic auth or external auth useless - all requests will be allowed. #access_by_lua_block { #} header_filter_by_lua_block { lua_ingress.header() plugins.run() } body_filter_by_lua_block { plugins.run() } log_by_lua_block { balancer.log() monitor.call() plugins.run() } port_in_redirect off; set $balancer_ewma_score -1; set $proxy_upstream_name "test1-echo2-80"; set $proxy_host $proxy_upstream_name; set $pass_access_scheme $scheme; set $pass_server_port $proxy_protocol_server_port; set $best_http_host $http_host; set $pass_port $pass_server_port; set $proxy_alternative_upstream_name ""; client_max_body_size 1m; proxy_set_header Host $best_http_host; # Pass the extracted client certificate to the backend # Allow websocket connections proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection $connection_upgrade; proxy_set_header X-Request-ID $req_id; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $best_http_host; proxy_set_header X-Forwarded-Port $pass_port; proxy_set_header X-Forwarded-Proto $pass_access_scheme; proxy_set_header X-Forwarded-Scheme $pass_access_scheme; proxy_set_header X-Scheme $pass_access_scheme; # Pass the original X-Forwarded-For proxy_set_header X-Original-Forwarded-For $http_x_forwarded_for; # mitigate HTTPoxy Vulnerability # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/ proxy_set_header Proxy ""; # Custom headers to proxied server proxy_connect_timeout 300s; proxy_send_timeout 300s; proxy_read_timeout 300s; proxy_buffering off; proxy_buffer_size 4k; proxy_buffers 4 4k; proxy_max_temp_file_size 1024m; proxy_request_buffering on; proxy_http_version 1.1; proxy_cookie_domain off; proxy_cookie_path off; # In case of errors try the next upstream server before returning an error proxy_next_upstream error timeout; proxy_next_upstream_timeout 0; proxy_next_upstream_tries 3; set $svr_ssl_client_cert ''; access_by_lua_block { ngx.var.svr_ssl_client_cert = ngx.var.ssl_client_escaped_cert or ngx.req.get_headers()["X-SSL-Client-Cert"] or "" } proxy_set_header X-SSL-Client-Cert $svr_ssl_client_cert; # Custom code snippet configured in the configuration configmap proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_pass http://upstream_balancer; proxy_redirect off; } # Custom code snippet configured in the configuration configmap # Prevent clickjacking attacks # Comment out this line if you want to make use of iframes add_header X-Frame-Options SAMEORIGIN; # Prevent "mime" based attacks add_header X-Content-Type-Options nosniff; # Prevent XSS attacks add_header X-XSS-Protection "1; mode=block"; } ## end server test.subdomain.domain.com ```
k8s-ci-robot commented 5 months ago

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
longwuyuan commented 5 months ago

/remove-kind bug

longwuyuan commented 5 months ago

/triage needs-information

Julien-Beezeelinx commented 5 months ago

Hi, thanks for the feedbacks.

longwuyuan commented 5 months ago 
% k explain ingress.spec.tls
GROUP:      networking.k8s.io
KIND:       Ingress
VERSION:    v1

FIELD: tls <[]IngressTLS>

DESCRIPTION:
    tls represents the TLS configuration. Currently the Ingress only supports a
    single TLS port, 443. If multiple members of this list specify different
    hosts, they will be multiplexed on the same port according to the hostname
    specified through the SNI TLS extension, if the ingress controller
    fulfilling the ingress supports SNI.
    IngressTLS describes the transport layer security associated with an
    ingress.

FIELDS:
  hosts <[]string>
    hosts is a list of hosts included in the TLS certificate. The values in this
    list must match the name/s used in the tlsSecret. Defaults to the wildcard
    host setting for the loadbalancer controller fulfilling this Ingress, if
    left unspecified.

  secretName    <string>
    secretName is the name of the secret used to terminate TLS traffic on port
    443. Field is left optional to allow TLS routing based on SNI hostname
    alone. If the SNI host in a listener conflicts with the "Host" header field
    used by an IngressRule, the SNI host is used for termination and value of
    the "Host" header is used for routing.

Also look at k explain ingress.spec.rules.host

/kind support

Julien-Beezeelinx commented 5 months ago

Hi,

longwuyuan commented 5 months ago

Somebody expert needs to check if sever-alias regexp complies with the below fact 

Kubernetes only accept expressions that comply with the RE2 engine syntax. It is possible that valid expressions accepted by NGINX cannot be used with ingress-nginx, because the PCRE library (used in NGINX) supports a wider syntax than RE2. See the [RE2 Syntax](https://github.com/google/re2/wiki/Syntax) documentation for differences.

cc @tao12345666333 @rikatz

/help

k8s-ci-robot commented 5 months ago

@longwuyuan: This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-help command.

In response to [this](https://github.com/kubernetes/ingress-nginx/issues/10953): >Somebody expert needs to check if sever-alias regexp complies with the below fact >``` >Kubernetes only accept expressions that comply with the RE2 engine syntax. It is possible that valid expressions accepted by NGINX cannot be used with ingress-nginx, because the PCRE library (used in NGINX) supports a wider syntax than RE2. See the [RE2 Syntax](https://github.com/google/re2/wiki/Syntax) documentation for differences. >``` > >cc @tao12345666333 @rikatz > >/help Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.