envoyproxy / gateway

Manages Envoy Proxy as a Standalone or Kubernetes-based Application Gateway
https://gateway.envoyproxy.io
Apache License 2.0
1.64k stars 354 forks source link

xds: use V4_PREFERRED dnsLookupFamily by default #4745

Open zirain opened 5 days ago

zirain commented 5 days ago

fixes: https://github.com/envoyproxy/gateway/issues/4744

zirain commented 5 days ago

cc @zetaab can you verify this on your env? hope it will work.

codecov[bot] commented 5 days ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 65.60%. Comparing base (78da42c) to head (c17c635).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #4745 +/- ## ========================================== - Coverage 65.61% 65.60% -0.02% ========================================== Files 211 211 Lines 31961 31961 ========================================== - Hits 20972 20968 -4 - Misses 9751 9753 +2 - Partials 1238 1240 +2 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


🚨 Try these New Features:

zirain commented 4 days ago

@zirain @arkodg I think V4_PREFERRED won't work for IPv6 env where the envoy pod only has an IPv6 address.

If V4_PREFERRED is specified, the DNS resolver will first perform a lookup for addresses in the IPv4 family and fallback to a lookup for addresses in the IPv6 family.

tested passed on an IPv6 only cluster.

https://github.com/envoyproxy/gateway/pull/4690/commits/3b265169ee3579c77a3dc9ab196e64fcefebc76e

zhaohuabing commented 4 days ago

3b26516

I mean "real" IPv6 only where the pod just has an IPv6 address and has no IPv4 address.

Does the pod in this test has an IPv4 address? If it only has an IPv6 address, it shouldn't be able to establish an connection to an IPv4 server.

zirain commented 4 days ago

3b26516

I mean "real" IPv6 only where the pod just has an IPv6 address and has no IPv4 address.

Does the pod in this test has an IPv4 address? If it only has an IPv6 address, it shouldn't be able to establish an connection to an IPv4 server.

it's IPv6 only cluster, not IPv6 first.

zhaohuabing commented 4 days ago

4744

I guess if AUTO(IPv6_prefered) didn't work for pod without IPv6 address ( like in https://github.com/envoyproxy/gateway/issues/4744), then IPv4_prefered may not work for pod without IPv4 address as well. I may be wrong, will test it myself.

zhaohuabing commented 4 days ago

This test verfied that V4_PREFERRED doesn't work with IPv6 only.

4745SP with a JWK configuration:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
  name: demo-api-jwt
spec:
  jwt:
    providers:
    - name: test
      remoteJWKS:
        uri: https://www.zhaohuabing.com/misc/jwks.json
  targetRef:
    group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: backend

DNS records for IPv4 and IPv6

➜  gateway git:(use/V4_PREFERRED) dig www.zhaohuabing.com     

;; ANSWER SECTION:
www.zhaohuabing.com.    247 IN  A   104.21.60.99
www.zhaohuabing.com.    247 IN  A   172.67.195.133

➜  gateway git:(use/V4_PREFERRED) dig AAAA www.zhaohuabing.com

;; ANSWER SECTION:
www.zhaohuabing.com.    244 IN  AAAA    2606:4700:3037::ac43:c385
www.zhaohuabing.com.    244 IN  AAAA    2606:4700:3034::6815:3c63

Test with IPv6 only, failed to fetch jwks

export IP_FAMILY=ipv6; export IMAGE_PULL_POLICY=IfNotPresent; make create-cluster kube-install-image kube-deploy

Envoy log

[2024-11-21 06:37:38.446][1][warning][jwt] [source/extensions/filters/http/jwt_authn/jwks_async_fetcher.cc:115] Jwks async fetching url=https://www.zhaohuabing.com/misc/jwks.json: failed

The jwks can be downloaded via curl --ipv6

curl --ipv6 -v https://www.zhaohuabing.com/misc/jwks.json
* Host www.zhaohuabing.com:443 was resolved.
* IPv6: 2606:4700:3034::6815:3c63, 2606:4700:3037::ac43:c385
* IPv4: (none)
*   Trying [2606:4700:3034::6815:3c63]:443...
* Connected to www.zhaohuabing.com (2606:4700:3034::6815:3c63) port 443
....
{ "keys":[ {"e":"AQAB","kid":"DHFbpoIUqrY8t2zpA2qXfCmr5VO5ZEr4RzHU_-envvQ","kty":"RSA","n":"xAE7eB6qugXyCAG3yhh7pkDkT65pHymX-P7KfIupjf59vsdo91bSP9C8H07pSAGQO1MV_xFj9VswgsCg4R6otmg5PV2He95lZdHtOcU5DXIg_pbhLdKXbi66GlVeK6ABZOUW3WYtnNHD-91gVuoeJT_DwtGGcp4ignkgXfkiEm4sw-4sfb4qdt5oLbyVpmW6x9cfa7vs2WTfURiCrBoUqgBo_-4WTiULmmHSGZHOjzwa8WtrtOQGsAFjIbno85jp6MnGGGZPYZbDAa_b3y5u-YpW7ypZrvD8BgtKVjgtQgZhLAGezMt0ua3DRrWnKqTZ0BJ_EyxOGuHJrLsn00fnMQ"}]}

Test with IPv4 only, succeeded

export IP_FAMILY=ipv4; export IMAGE_PULL_POLICY=IfNotPresent; make create-cluster kube-install-image kube-deploy
[2024-11-21 06:32:39.285][1][debug][filter] [source/extensions/filters/http/common/jwks_fetcher.cc:92] onSuccess: fetch pubkey [uri = https://www.zhaohuabing.com/misc/jwks.json]: succeeded
zirain commented 4 days ago

can you passed the test with IPv4_only on a IPv6 only cluster?

zhaohuabing commented 3 days ago

envoyProxy resource IPFamily is only meant for listener and Envoy Proxy fleet Service spec

BackendRef IPFamily determines cluster setting

Are we planning to introduce another "BackendRef IPFamily" configuration knob? Even though they have different meaning, typically the value of listener IP famaily and the Backend IP family for in-cluster services should be the same.