Support IP whitelisting

[richarddli]

Feature request based on this comment: https://medium.com/@futuredon/hi-richard-41fd2a6d35c2

The default NGINX ingress supports IP whitelisting. See http://container-solutions.com/kubernetes-quick-tip/ for details. We could do this too.

gz#438

[plombardi89]

Quite possibly non-trivial. Nginx has support for IP black and white lists using allow and deny rules when it's configured with the http_access module which is part of the default install.

There's probably a couple paths we could take and they may involve contributing upstream to Envoy. I didn't see anything to indicate this exists already.

[richarddli]

I'm rereading the Container Solutions blog post, and I wonder how the NGINX controller does this in a real-world situation where you use a LoadBalancer. The functionality seems only useful if you're deploying as a NodePort -- otherwise, you really want to do something with X-Forwarded-For (which we support via regex_header).

[plombardi89]

Nginx has configuration directives to handle X-Forwarded-For... we used it back in the MCP days. My guess is it configures thing similarly to this (it wouldn't be that hard to find out either):

upstream backend {
  server 127.0.0.1:52689;
}

server {
  listen 80      proxy_protocol;
  listen [::]:80 proxy_protocol;

  server_name backend_proxy;

  real_ip_header   X-Forwarded-For;
  set_real_ip_from 10.0.0.0/16;

  location / {
    proxy_pass          http://backend;
    proxy_http_version  1.1;
    proxy_set_header    X-Real-IP       $proxy_protocol_addr;
    proxy_set_header    X-Forwarded-For $proxy_protocol_addr;
    #proxy_set_header    X-Real-IP $remote_addr;
    #proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header    Host $http_host;
  }

  location /rtp {
    proxy_pass          http://backend;
    proxy_http_version  1.1;
    proxy_set_header    X-Real-IP       $proxy_protocol_addr;
    proxy_set_header    X-Forwarded-For $proxy_protocol_addr;
    #proxy_set_header    X-Real-IP $remote_addr;
    #proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header    Host $http_host;
    proxy_set_header    Upgrade $http_upgrade;
    proxy_set_header    Connection "upgrade";
  }
}

#
# This server block is only needed if you need Proxy Protocol support and use a classic
# ELB with TCP+TLS because of web sockets. The load balancer health check mechanism does not
# actually send the requests through the load balancer in front of the service. This means the
# health check traffic that reaches the server does not adhere to the proxy protocol and when
# Nginx is configured to expect proxy protocol requests then things do not work as expected.
#
# Read more about Proxy Protocol on Amazon:
#   - http://docs.aws.amazon.com/elasticloadbalancing/latest/classic/enable-proxy-protocol.html
#

server {
  listen 81;
  listen [::]:81;
  server_name backend_health_proxy;

  location = /health {
    proxy_pass          http://backend;
      proxy_http_version  1.1;
    proxy_set_header    X-Real-IP $remote_addr;
    proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header    Host $http_host;
  }
}

[ghost]

+1 on this ;)

[suulperi]

I would see IP whitelisting as a wonderful feature. I can see it useful for limiting APIs for certains source ip addresses. +1 for this one.

[richarddli]

Example use case: Add ACL for specific subnets to a specific service. For example, only allow 10.3.5.x to access Jenkins.

[sbrichardson]

I was just checking out the Envoy docs, you could potentially use the rate limit service, similar to the auth service? Or just use the auth service also, since it's really just a pass/fail check. Delegating to a service would allow you eliminate entire countries/subnets or other magic, know bad actors, 3rd party sync services etc. That's more where we are, so a bit out of scope.

I could see their being benefit to adding a basic whitelist/blacklist support built in so that you don't need to roll your own service.

[kflynn]

Note that there's a V2-only RBAC filter, too (https://www.envoyproxy.io/docs/envoy/v1.7.0/configuration/http_filters/rbac_filter) that appears to be able to do this.

[mtbdeano]

I would like to add the loadBalancerSourceRanges key to the LoadBalancer resource created by ambassador, this allows AWS ELB's to be much more secure. https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/#restrict-access-for-loadbalancer-service Does this fit into this issue? Or is it separate?

[plombardi89]

@mtbdeano you can just use a custom Service manifest with that attribute added to route into Ambassador. This is about blocking IP's upon request by Ambassador/Envoy. That would be blocking IP's at the hardware / cloud firewall level.

[gertvdijk]

Note that there's a V2-only RBAC filter, too (https://www.envoyproxy.io/docs/envoy/v1.7.0/configuration/http_filters/rbac_filter) that appears to be able to do this.

Looks like more of the API is described here config.rbac.v2alpha.Principal field source_ip.

It seems that this is more of a global-level RBAC control, not really per-route. I'm hoping that Ambassador's implementation in the future will allow me to authorize IPs per route, so that I could have the rules in the Service annotations. (Having to duplicate the URIs for the matches in RBAC rules separately as Envoy implements it would seem very tedious to me.)

[caiohasouza]

+1

[vaibhavrtk]

+1

[youalreadydid]

+1

[ekarmazin]

+1

[larrywest42]

Alas, Envoy's config.rbac.v2.Principal.source_ip field, even in v1.12.0, is a core.CidrRange, which holds a single CIDR (address+len for a single subnet).

So, unless I'm misreading something, this would only work for white-listing (or black-listing) a single network block.

[ediskandarov]

config.rbac.v2.Principal.source_ip does not work if envoy is behind a proxy and real client IP detection is based on X-Forwarded-For header

source_ip always contains ip src address value

[LukeShu]

Note that there's a V2-only RBAC filter, too (https://www.envoyproxy.io/docs/envoy/v1.7.0/configuration/http_filters/rbac_filter) that appears to be able to do this.

The current link for that is: https://www.envoyproxy.io/docs/envoy/v1.12.0/configuration/http/http_filters/rbac_filter

[ghost]

Just jumping in to say with istio you can set the ingress externalTrafficPolicy to local, this will get the originating IP(the callers public ip)

kubectl edit svc istio-ingressgateway -n istio-system
#Change externalTrafficPolicy: Local

Then set an Authorization policy as follows.

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: ingress-policy
  namespace: istio-system
spec:
  selector:
    matchLabels:
      app: istio-ingressgateway
  action: ALLOW
  rules:
    - from:
        - source:
            ipBlocks: ["1.1.1.1/32"]

[Guillermogsjc]

Hi, is there any developed way to filter IP sources at the Mapping object level ?

The use case is pretty clear: to filter ip sources based on the service being proxied in each Mapping. It would be so nice to have this kind of feature... So far the best I know is to do it at Module level globally applying it for the whole ingress object.

If there is not a direct way, is maybe a hacky way of use RateLimit object to limit to 0 IP not in a list?

Regards.

[irl-segfault]

@Guillermogsjc would like to know this as well. strong use case.

[Tarasovych]

I'd also like to have an opportunity to block or allow IP sources at the mapping level. Or attach module to path

[juanjoku]

In our clusters we also require this functionality.

I understand that the "RBAC filter" Envoy feature can be exploited by Ambassador, and include a white/black-list in the configuration of the Mappings and TCPMappings.

In case of being behind a load balancer.... won't it be enough to enable the proxy-protocol?

[juanjoku]

Are you planning to include this functionality in a future release? It would be very appreciated!

[richarddli]

You could definitely use RateLImit or create an External Filter as an interim solution. I don't know if this is something we'll prioritize super-highly in the near future, but appreciate all the feedback.

[juanjoku]

Thanks, Richard. We were just considering that option (ratelimit) :-)

[m-yosefpor]

This is also a required feature for our organization, as different mappings require their own whitelisting rules.

I think sth like what have been implemented for modules (as described here https://www.getambassador.io/docs/edge-stack/latest/topics/running/ambassador/#ip-allow-and-deny) would be very useful for mapping objects as well.

[wadexu007]

+1.

Currently my workaround is install two Emissary in one cluster. one Emissary facing to internet. another one facing to ip_allow list. (need set another ambassador_id in second Emissary to avoid conflict)

[wadexu007]

I'd also like to have an opportunity to block or allow IP sources at the mapping level. Or attach module to path

Totally agree.