envoyproxy / envoy

Cloud-native high-performance edge/middle/service proxy
https://www.envoyproxy.io
Apache License 2.0
24.33k stars 4.71k forks source link

Filter chain match by `sourcePrefixRanges` is not working correctly with CIDR `0.0.0.0/0` #34299

Open JoMC98 opened 2 months ago

JoMC98 commented 2 months ago

Title: Filter chain match by sourcePrefixRanges is not working correctly with CIDR 0.0.0.0/0

Description:

In our LDS configuration, we are configuring 2 different filter chains. Both filter chains have the same configuration and log the Filter Chain Name field, which allows us to see which filter chain is being applied.

  • The first one has no FilterChainMatch and applies by default.
  • The second one has a FilterChainMatch for sourcePrefixRanges.

If we configure the second FilterChainMatch with the range sourcePrefixRanges 127.0.0.0/24, it works correctly:

  • If Source IP is 127.0.0.1, it applies the second filter chain:
    {"downstreamIp":"127.0.0.1","filterChain":"filter_chain_with_match"}
    {"downstreamIp":"127.0.0.1","filterChain":"filter_chain_with_match"}
  • If Source IP is not on 127.0.0.0/24 range (10.97.145.78), it applies the first filter chain:
    {"downstreamIp":"10.97.145.78","filterChain":"default"}
    {"downstreamIp":"10.97.145.78","filterChain":"default"}

If we configure the second FilterChainMatch with the range sourcePrefixRanges 0.0.0.0/0, it should always apply the second filter chain, regardless of the source IP, but it does not work correctly. Each time we start Envoy with the same LDS config, either the first or the second one is being applied randomly:

{"downstreamIp":"10.97.145.78","filterChain":"default"}
{"downstreamIp":"127.0.0.1","filterChain":"default"}
{"downstreamIp":"10.97.145.78","filterChain":"filter_chain_with_match"}
{"downstreamIp":"127.0.0.1","filterChain":"filter_chain_with_match"}

Config that works with 127.0.0.0/24:

resources:
  - name: listener_http
    address:
      socketAddress:
        address: 0.0.0.0
        portValue: 8081
    filterChains:
      - name: default
        filters:
          - name: envoy.filters.network.http_connection_manager
            typedConfig:
              statPrefix: ingress_http
              routeConfig:
                name: local_route
                virtualHosts:
                  - name: local_service
                    domains:
                      - '*'
                    routes:   
                      - match:
                          prefix: /
                        route:
                          cluster: service_upstream
                          timeout: 60s
              httpFilters:                
                - name: envoy.filters.http.router
                  typedConfig:
                    '@type': >-
                      type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
              accessLog:
                - name: envoy.access_loggers.stdout
                  typedConfig:
                    logFormat:
                      jsonFormat:
                        downstreamIp: '%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%'
                        filterChain: default
                      contentType: json_format
                    '@type': >-
                      type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
              '@type': >-
                type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
      - name: filter_chain_with_match
        filterChainMatch:
          sourcePrefixRanges:
            - addressPrefix: 127.0.0.0
              prefixLen: 24
        filters:
          - name: envoy.filters.network.http_connection_manager
            typedConfig:
              statPrefix: ingress_http
              routeConfig:
                name: local_route
                virtualHosts:
                  - name: local_service
                    domains:
                      - '*'
                    routes:   
                      - match:
                          prefix: /
                        route:
                          cluster: service_upstream
                          timeout: 60s
              httpFilters:                
                - name: envoy.filters.http.router
                  typedConfig:
                    '@type': >-
                      type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
              accessLog:
                - name: envoy.access_loggers.stdout
                  typedConfig:
                    logFormat:
                      jsonFormat:
                        downstreamIp: '%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%'
                        filterChain: filter_chain_with_match
                      contentType: json_format
                    '@type': >-
                      type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
              '@type': >-
                type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
    '@type': type.googleapis.com/envoy.config.listener.v3.Listener

Config that NOT works with 0.0.0.0/0:

resources:
  - name: listener_http
    address:
      socketAddress:
        address: 0.0.0.0
        portValue: 8081
    filterChains:
      - name: default
        filters:
          - name: envoy.filters.network.http_connection_manager
            typedConfig:
              statPrefix: ingress_http
              routeConfig:
                name: local_route
                virtualHosts:
                  - name: local_service
                    domains:
                      - '*'
                    routes:   
                      - match:
                          prefix: /
                        route:
                          cluster: service_upstream
                          timeout: 60s
              httpFilters:                
                - name: envoy.filters.http.router
                  typedConfig:
                    '@type': >-
                      type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
              accessLog:
                - name: envoy.access_loggers.stdout
                  typedConfig:
                    logFormat:
                      jsonFormat:
                        downstreamIp: '%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%'
                        filterChain: default
                      contentType: json_format
                    '@type': >-
                      type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
              '@type': >-
                type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
      - name: filter_chain_with_match
        filterChainMatch:
          sourcePrefixRanges:
            - addressPrefix: 0.0.0.0
              prefixLen: 0
        filters:
          - name: envoy.filters.network.http_connection_manager
            typedConfig:
              statPrefix: ingress_http
              routeConfig:
                name: local_route
                virtualHosts:
                  - name: local_service
                    domains:
                      - '*'
                    routes:   
                      - match:
                          prefix: /
                        route:
                          cluster: service_upstream
                          timeout: 60s
              httpFilters:                
                - name: envoy.filters.http.router
                  typedConfig:
                    '@type': >-
                      type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
              accessLog:
                - name: envoy.access_loggers.stdout
                  typedConfig:
                    logFormat:
                      jsonFormat:
                        downstreamIp: '%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%'
                        filterChain: filter_chain_with_match
                      contentType: json_format
                    '@type': >-
                      type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
              '@type': >-
                type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
    '@type': type.googleapis.com/envoy.config.listener.v3.Listener
zuercher commented 2 months ago

What would you expect to happen in the case where you have a default matcher and a matcher for 0.0.0.0/0? Both will match all requests.

In practice, after validation the default matcher is implemented as a matcher against source ip 0.0.0.0/0 or ::/0 (if IPv6 is enabled). The bug here is that your config should be rejected because the two filter chains have duplicate matchers. Perhaps we should allow this case as a way to handle ipv4 vs ipv6 addresses in filter chain match.

JoMC98 commented 1 month ago

Hi Stephan, thanks for the reply.

True, both matchers match, but as the second one has a filter, even if it is 0.0.0.0/0, isn't it more specific than the default one which has nothing?

I tell you our scenario:

The default filter chain has JWT Authentication and RBAC Authorization filters applied. In our network, we want certain IP ranges not to have these filters applied, so we create the second filter chain, matched by sourcePrefixRanges. In a normal scenario, this works fine.

Now, we have proposed to use this use case for a Plan B scenario, where we need all traffic to reach the upstream without applying the JWT and RBAC filters. So, in a simple way in our infrastructure, we can change the range of the sourcePrefixRanges and we had thought of changing it to 0.0.0.0/0 so that all traffic would go through the second filter chain.

How could we do this taking into account what you said about the 0.0.0.0/0 range?