search

zufardhiyaulhaq / istio-ratelimit-operator

Automatically create global & local rate limit in Istio, support EnvoyFilter versioning, ratelimit metrics, shadow mode, and ratelimit based on route!
MIT License
39 stars 9 forks source link

Rate limiting not happening #36

Closed Abhishek627 closed 1 year ago

Abhishek627 commented 1 year ago

I followed the example for global rate limit and see no keys in redis and no logs for rate limiting in rate limit service as well.

I'm unclear on what value it required at https://github.com/zufardhiyaulhaq/istio-ratelimit-operator/blob/c8563fbd4dc7acd5d55db7de02f7ebdddab4e780/examples/global/gateway/ratelimitservice/globalratelimit.yaml#L13

If below is my virtual service and i'm giving selector like this. Is this correct ? 

selector:
    vhost: "abhishek-sample.com:443"
    route: "productpage"
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: bookinfo
  namespace:  test
spec:
  gateways:
  - istio-gateway
  - istio-ingress/default-gw
  hosts:
  - abhishek-sample.com
  http:
  - match:
    - uri:
        exact: /productpage
    route:
    - destination:
        host: productpage
        port:
          number: 9080
Abhishek627 commented 1 year ago

My rate limit config is generated like below:

apiVersion: v1
data:
  config.yaml: |
    domain: public-gateway
    descriptors:
    - key: method
      descriptors:
      - key: machineid
        descriptors:
        - key: route
          value: productpage
          rate_limit:
            unit: minute
            requests_per_unit: 5
    - key: path
      descriptors:
      - key: route
        value: productpage
        rate_limit:
          unit: minute
          requests_per_unit: 5
kind: ConfigMap

And i'm expecting it to rate limit on https://abhishek-sample.com/productpage after 5 requests.

Abhishek627 commented 1 year ago

service logs:

time="2023-04-12T15:19:37Z" level=warning msg="statsd is not in use"
time="2023-04-12T15:19:37Z" level=info msg="Tracing disabled"
time="2023-04-12T15:19:37Z" level=warning msg="connecting to redis on redis.istio-system.svc.cluster.local:6379 with pool size 10"
time="2023-04-12T15:19:37Z" level=warning msg="Listening for debug on '0.0.0.0:6070'"
time="2023-04-12T15:19:37Z" level=warning msg="Listening for gRPC on '0.0.0.0:8081'"
time="2023-04-12T15:19:37Z" level=warning msg="Listening for HTTP on '0.0.0.0:8080'"
time="2023-04-12T15:22:24Z" level=warning msg="runtime: error processing /data/ratelimit/config/..data_tmp: lstat /data/ratelimit/config/..data_tmp: no such file or directory"
time="2023-04-12T15:34:41Z" level=warning msg="runtime: error processing /data/ratelimit/config/..2023_04_12_15_22_24.172543879: open /data/ratelimit/config/..2023_04_12_15_22_24.172543879: no such file or directory"
time="2023-04-12T15:36:02Z" level=warning msg="runtime: error processing /data/ratelimit/config/..2023_04_12_15_34_41.560530269: readdirent /data/ratelimit/config/..2023_04_12_15_34_41.560530269: no such file or directory"
time="2023-04-12T15:45:05Z" level=warning msg="runtime: error processing /data/ratelimit/config/..data_tmp: lstat /data/ratelimit/config/..data_tmp: no such file or directory"
Abhishek627 commented 1 year ago

@zufardhiyaulhaq Any input on this please ?

zufardhiyaulhaq commented 1 year ago

@Abhishek627 can you give the configuration of GlobalRateLimitConfig, GlobalRateLimit, and RateLimitService? also, what is the Istio version & istio-ratelimit-operator you currently running?

Abhishek627 commented 1 year ago

@zufardhiyaulhaq Istio version is 1.13.5 and i'm using latest istio-ratelimit-operator from master branch. Productpage is a virtual-service in namespace test with match on prefix /productpage route.

apiVersion: ratelimit.zufardhiyaulhaq.com/v1alpha1
kind: GlobalRateLimit
metadata:
  name: productpage-route2
  namespace: istio-system
spec:
  config: "istio-public-gateway"
  selector:
    vhost: "abhishek-sample.com:443"
    route: "productpage"
  matcher:
  - request_headers:
      header_name: ":path"
      descriptor_key: "path"
  limit:
    unit: minute
    requests_per_unit: 5
  shadow_mode: false
apiVersion: ratelimit.zufardhiyaulhaq.com/v1alpha1
kind: GlobalRateLimitConfig
metadata:
  name: istio-public-gateway
  namespace: istio-system
spec:
  type: "gateway"
  selector:
    labels:
      "istio": "ingressgateway"
      # "app": "istio-public-gateway"
    istio_version:
      - "1.13"
    sni: "abhishek-sample.com"
  ratelimit:
    spec:
      domain: "public-gateway"
      failure_mode_deny: false
      timeout: "10s"
      service:
        type: "service"
        name: "public-gateway-ratelimit-service"
apiVersion: ratelimit.zufardhiyaulhaq.com/v1alpha1
kind: RateLimitService
metadata:
  name: public-gateway-ratelimit-service
  namespace: istio-system
spec:
  kubernetes:
    replica_count: 1
    auto_scaling:
      max_replicas: 3
      min_replicas: 1
    resources:
      limits:
        cpu: "256m"
        memory: "256Mi"
      requests:
        cpu: "128m"
        memory: "128Mi"     
  backend:
    redis:
      type: "single"
      url: "redis.istio-system.svc.cluster.local:6379"
zufardhiyaulhaq commented 1 year ago

@Abhishek627 can you give the gateway & virtualservice configuration for abhishek-sample.com? also can you check if envoyfilter is generated on istio-system namespace?

Abhishek627 commented 1 year ago 
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: bookinfo
  namespace: test
spec:
  hosts:
  - abhishek-sample.com
  gateways:
  - test-istio-gateway
  - istio-ingress/default-gw
  http:
  - match:
    - uri:
        exact: /productpage
    - uri:
        prefix: /static
    - uri:
        exact: /login
    - uri:
        exact: /logout
    - uri:
        prefix: /api/v1/products
    route:
    - destination:
        host: productpage
        port:
          number: 9080
 ---
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  annotations:
    package: sample
  name: test-istio-gateway
  namespace: test
spec:
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - abhishek-sample.com
    port:
      name: https-443
      number: 443
      protocol: HTTPS
    tls:
      credentialName: ingress-gateway-cert
      mode: SIMPLE
  - hosts:
    - abhishek-sample.com
    port:
      name: http-80
      number: 80
      protocol: HTTP
    tls:
      httpsRedirect: true
Abhishek627 commented 1 year ago

It generated these 2 envoyfilters in istio-system ns.

---
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  creationTimestamp: "2023-04-12T15:19:44Z"
  generation: 2
  labels:
    istio/version: "1.13"
  name: productpage-route2-1.13
  namespace: istio-system
  ownerReferences:
  - apiVersion: ratelimit.zufardhiyaulhaq.com/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: GlobalRateLimit
    name: productpage-route2
    uid: a9903e4d-7b0f-4b4a-b6b6-c278f4671ee2
  resourceVersion: "34560058"
  uid: 2e09e27b-fe3d-404d-bdb2-9346c7451cbf
spec:
  configPatches:
  - applyTo: HTTP_ROUTE
    match:
      context: GATEWAY
      proxy:
        proxyVersion: ^1\.13.*
      routeConfiguration:
        vhost:
          name: abhishek-sample.com:443
          route:
            name: productpage
    patch:
      operation: MERGE
      value:
        route:
          rate_limits:
          - actions:
            - request_headers:
                descriptor_key: path
                header_name: :path
  workloadSelector:
    labels:
      istio: ingressgateway
---
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  creationTimestamp: "2023-04-12T15:19:45Z"
  generation: 1
  labels:
    istio/version: "1.13"
  name: istio-public-gateway-1.13
  namespace: istio-system
  ownerReferences:
  - apiVersion: ratelimit.zufardhiyaulhaq.com/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: GlobalRateLimitConfig
    name: istio-public-gateway
    uid: 13aab623-6c89-4415-89e0-65ed31fb65dd
  resourceVersion: "34537335"
  uid: 4ddfc80c-2f07-4157-9e1e-767bfb9f1966
spec:
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: GATEWAY
      listener:
        filterChain:
          filter:
            name: envoy.filters.network.http_connection_manager
            subFilter:
              name: envoy.filters.http.router
          sni: abhishek-sample.com
      proxy:
        proxyVersion: ^1\.13.*
    patch:
      operation: INSERT_BEFORE
      value:
        name: envoy.filters.http.ratelimit
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
          domain: public-gateway
          failure_mode_deny: false
          rate_limit_service:
            grpc_service:
              envoy_grpc:
                cluster_name: istio-public-gateway
              timeout: 10s
            transport_api_version: V3
          timeout: 10s
  - applyTo: CLUSTER
    match:
      cluster:
        service: public-gateway-ratelimit-service.istio-system.svc.cluster.local
      context: GATEWAY
      proxy:
        proxyVersion: ^1\.13.*
    patch:
      operation: ADD
      value:
        connect_timeout: 10s
        http2_protocol_options: {}
        lb_policy: ROUND_ROBIN
        load_assignment:
          cluster_name: istio-public-gateway
          endpoints:
          - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: public-gateway-ratelimit-service.istio-system.svc.cluster.local
                    port_value: 8081
        name: istio-public-gateway
        type: STRICT_DNS
  workloadSelector:
    labels:
      istio: ingressgateway

and below are the pods and services in istio-system ns.

service/my-istio-ratelimit-operator-controller-manager-metrics-service   ClusterIP   172.20.20.161    <none>        8443/TCP                                                5d20h
service/public-gateway-ratelimit-service                                 ClusterIP   172.20.186.168   <none>        8080/TCP,8081/TCP,6070/TCP,9102/TCP,9125/TCP,9125/UDP   5d19h
pod/my-istio-ratelimit-operator-controller-manager-56ff8f9b7b-7vr42   2/2     Running   0          5d20h
pod/public-gateway-ratelimit-service-59c5c677b6-grkng                 1/1     Running   0          5d18h
zufardhiyaulhaq commented 1 year ago

@Abhishek627 

apiVersion: ratelimit.zufardhiyaulhaq.com/v1alpha1
kind: GlobalRateLimit
metadata:
  name: productpage-route2
  namespace: istio-system
spec:
  config: "istio-public-gateway"
  selector:
    vhost: "abhishek-sample.com:443"
    route: "productpage"
.
.
.

there must be a productpage route name defined on your virtualservice. for example

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: bookinfo
  namespace: test
spec:
  hosts:
  - abhishek-sample.com
  gateways:
  - test-istio-gateway
  - istio-ingress/default-gw
  http:
  - name: productpage
     match:
    - uri:
        exact: /productpage
    route:
    - destination:
        host: productpage
        port:
          number: 9080

https://istio.io/latest/docs/reference/config/networking/virtual-service/#HTTPRoute image

Abhishek627 commented 1 year ago

Ohh ! I'll give it a shot. Thank you @zufardhiyaulhaq !! 🥇