search

projectcontour / contour

Contour is a Kubernetes ingress controller using Envoy proxy.
https://projectcontour.io
Apache License 2.0
3.73k stars 680 forks source link

I would like to be able to specify the host header when using External Authz. #6167

Closed kahirokunn closed 3 weeks ago

kahirokunn commented 9 months ago

If you place an additional envoy in front of the authentication server, it will not be able to deliver traffic to the authentication server without a host header. Therefore, we would like to be able to set the host header in the External Authz.

tsaarni commented 9 months ago

Would it be this one for gRPC / HTTP2 https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/grpc_service.proto#envoy-v3-api-msg-config-core-v3-grpcservice-envoygrpc ?

authority

(string) The :authority header in the grpc request. If this field is not set, the authority header value will be cluster_name.

We set it here with the same value as TLS SNI https://github.com/projectcontour/contour/blob/39a7d2002389792660c4ee7ffb2806b53e0c334d/internal/envoy/v3/listener.go#L928-L943

The value originates from ExtensionService.spec.validation.subjectName, or subjectNames[0] in future (link).

kahirokunn commented 9 months ago

Thank you! I am hoping to set a host header when communicating to the authority service.

I think ExtensionService.spec.validation.subjectName is an option for certificate validation, but if I enable it, does it mean that the value in ExtensionService.spec.validation.subjectName will be set as Is this so that the value in ExtensionService.spec.validation.subjectName will be set as the host header?

kahirokunn commented 9 months ago

I am not very familiar with how to set up envoy itself

tsaarni commented 9 months ago

I have not tried myself but maybe you could try setting subjectName and see how it goes.

ExtensionService.spec.validation.subjectName is bit overloaded. Besides certificate validation it is used as server name for TLS SNI extension to signal the server which virtualhost the TLS connection is meant for, and it is also used for :authority header. External authorization uses gRPC, which uses uses HTTP/2, where Host: header is replaced with :authority pseudo-header - similarly to signal the server which virtualhost the request is meant for. The parameter gets programmed to the gRPC client within Envoy that sends the requests towards external authorization server.

kahirokunn commented 9 months ago

I see, that is helpful! In that case, does the ExternalAuthz have to TLS? My k8s cluster's internal communication is mainly plain h2c, not using tls.

tsaarni commented 9 months ago

Yes, I think it has to be TLS for the subjectName approach. It looks like there would be an error upstream TLS validation not supported for h2c protocol when h2c is used.

I wonder if there is other approach to set the authority field for gRPC, without explicitly adding a new API?

While writing #6098 I began thinking how would I derive hostname/servername automatically. One approach could be simply to use (non-namespaced) Service name. Strictly speaking it should be FQDN with full domain. Envoy and upstream service are likely running in different namespace and technically from DNS perspective plain hostname would not even work. But FQDN might be difficult to derive and even the upstream servers likely are unaware of their own full domain name, so I think people might be inclined to simply use the unqualified service name.

In your case, what would you use as the hostname?

kahirokunn commented 9 months ago
  1. envoy and authority service exist in different namespaces.
  2. authority service is currently running under deployment and connected via h2c, which is unwilling. I would like to start it as a knative service, but I have a problem with host headers not reaching it.
kahirokunn commented 9 months ago

I wish h2c would set the authority field as well...

tsaarni commented 9 months ago

I'm unfortunately not familiar with knative, but do you know what exact :authority/Host: header value it expects for routing? Just short service name like myextauth or FQDN like myextauth.mynamespace.svc.cluster.local?

kahirokunn commented 9 months ago

Just short service name like myextauth or FQDN like myextauth.mynamespace.svc.cluster.local

Both are supported.

kahirokunn commented 9 months ago

For the knative gateway I use "contour". That is, it is just envoy and supports all host header formats. All you need to do is to send a host header indicating the authority service.

tsaarni commented 9 months ago

Currently the TLS approach discussed above is the only one available.

Maybe it could be a small code change to automatically set the header using the Service name (just hostname, not FQDN) - at least when there is one upstream service. It can be argued that using unqualified name is not technically correct. Currently the default according to Envoy docs is the Envoy cluster name (generated by Contour) which is not correct either.

There is a chance that unqualified hostname might not work and explicit configuration option would be required, but it would be bigger code change to introduce a new API in ExtensionService to let user to set it.

kahirokunn commented 9 months ago

Thank you! I will try to do it once in the form of using a certificate. However, it would be more convenient and resource-saving to be able to use h2c without a certificate, so we will continue to request such a feature.

github-actions[bot] commented 6 months ago

The Contour project currently lacks enough contributors to adequately respond to all Issues.

This bot triages Issues according to the following rules:

You can:

Please send feedback to the #contour channel in the Kubernetes Slack

kahirokunn commented 6 months ago

Keep

lsjostro commented 6 months ago

Can't you use :authority header which is what envoy sends to ext authz service? We use that to match the virtualhost.fqdn in httpproxy resource in our authserver.

Example: https://github.com/shelmangroup/envoy-oidc-authserver/blob/1c2a8e64936e03e04dba110de23b17c44cdaf0ff/run/k8s/manifests/oidc-providers.yaml#L13

tsaarni commented 6 months ago

I believe :authority was being used, even though host header was mentioned. The problem is that when :authority is not explicitly configured for Envoy's gRPC client, the default fallback behaviour is to use internally generated identifier cluster_name as :authority (link). In case of Contour, the generated cluster_name identifiers do not resemble hostnames and are therefore useless for vhost based routing (of the gRPC request itself, not the request-to-be-authorized).

My understanding is that the ask in this issue is to provide a way for user to set the value of :authority in case of h2c protocol. Currently there is no API for that.

If h2 would be used, then user can define :authority in a roundabout way by setting the value in ExtensionService.spec.validation.subjectNames[0]. The field is intended for validating the DNS name in server cert, but it happens to be also used by Contour to set the TLS SNI servername in TLS handshake and :authority in the HTTP/2 request.

kahirokunn commented 6 months ago

I have tried authority by launching envoy directly, but I could not confirm that it works. That is because I am not used to writing envoy config. I would really like to have this feature if hostheader is configured.

github-actions[bot] commented 4 months ago

The Contour project currently lacks enough contributors to adequately respond to all Issues.

This bot triages Issues according to the following rules:

You can:

Please send feedback to the #contour channel in the Kubernetes Slack

kahirokunn commented 4 months ago

keep

github-actions[bot] commented 2 months ago

The Contour project currently lacks enough contributors to adequately respond to all Issues.

This bot triages Issues according to the following rules:

You can:

Please send feedback to the #contour channel in the Kubernetes Slack

github-actions[bot] commented 3 weeks ago

The Contour project currently lacks enough contributors to adequately respond to all Issues.

This bot triages Issues according to the following rules:

You can:

Please send feedback to the #contour channel in the Kubernetes Slack