search

emissary-ingress / emissary

open source Kubernetes-native API gateway for microservices built on the Envoy Proxy
https://www.getambassador.io
Apache License 2.0
4.32k stars 685 forks source link

Rate Limit example does not work #5584

Closed grumpymatt closed 4 months ago

grumpymatt commented 4 months ago

Describe the bug The basic rate limiting example does not work with emissary 3.9.1 https://www.getambassador.io/docs/emissary/latest/topics/using/rate-limits

To Reproduce Steps to reproduce the behavior:

  1. Install emissary 3.9.1 via https://www.getambassador.io/docs/emissary/latest/tutorials/getting-started
  2. Install quote service
  3. Install rate limit example
  4. Update quote service mapping with rate limit labels: https://github.com/emissary-ingress/ratelimit-example/blob/main/k8s/quote-mapping-ratelimited.yaml ( api version has to be changed to v2)

Expected behavior The rate limit example should install and work according to documentation

Versions (please complete the following information):

Additional context emissary-ingress shows this error after updating the quote Mappings



[2024-02-28 14:32:19.397][57][warning][misc] [source/common/protobuf/message_validator_impl.cc:21] Deprecated field: type envoy.config.route.v3.HeaderMatcher Using deprecated option 'envoy.config.route.v3.HeaderMatcher.exact_match' from file route_components.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/version_history/version_history for details. If continued use of this field is absolutely necessary, see https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime#using-runtime-overrides-for-deprecated-features for how to apply a temporary and highly discouraged override.
[2024-02-28 14:32:19.397][57][warning][misc] [source/common/protobuf/message_validator_impl.cc:21] Deprecated field: type envoy.config.route.v3.HeaderMatcher Using deprecated option 'envoy.config.route.v3.HeaderMatcher.exact_match' from file route_components.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/version_history/version_history for details. If continued use of this field is absolutely necessary, see https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime#using-runtime-overrides-for-deprecated-features for how to apply a temporary and highly discouraged override.
[2024-02-28 14:32:19.397][57][warning][misc] [source/common/protobuf/message_validator_impl.cc:21] Deprecated field: type envoy.config.route.v3.HeaderMatcher Using deprecated option 'envoy.config.route.v3.HeaderMatcher.exact_match' from file route_components.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/version_history/version_history for details. If continued use of this field is absolutely necessary, see https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime#using-runtime-overrides-for-deprecated-features for how to apply a temporary and highly discouraged override.
[2024-02-28 14:32:19.398][57][critical][main] [source/server/config_validation/server.cc:67] error initializing configuration '/ambassador/snapshots/econf-tmp.json': Proto constraint validation failed (HttpConnectionManagerValidationError.RouteConfig: embedded message failed validation | caused by RouteConfigurationValidationError.VirtualHosts[0]: embedded message failed validation | caused by VirtualHostValidationError.Routes[4]: embedded message failed validation | caused by RouteValidationError.Route: embedded message failed validation | caused by RouteActionValidationError.RateLimits[0]: embedded message failed validation | caused by RateLimitValidationError.Actions[0]: embedded message failed validation | caused by ActionValidationError.RequestHeaders: embedded message failed validation | caused by RequestHeadersValidationError.HeaderName: value length must be at least 1 characters): stat_prefix: "ingress_http"
route_config {
  virtual_hosts {
    name: "emissary-ingress-listener-8080-*"
    domains: "*"
    routes {
      match {
        prefix: "/ambassador/v0/check_ready"
        case_sensitive {
          value: true
        }
        headers {
          name: "x-forwarded-proto"
          exact_match: "https"
        }
        runtime_fraction {
          default_value {
            numerator: 100
          }
          runtime_key: "routing.traffic_shift.cluster_127_0_0_1_8877_emissary"
        }
      }
      route {
        cluster: "cluster_127_0_0_1_8877_emissary"
        prefix_rewrite: "/ambassador/v0/check_ready"
        timeout {
          seconds: 10
        }
      }
    }
    routes {
      match {
        prefix: "/ambassador/v0/check_ready"
        case_sensitive {
          value: true
        }
        runtime_fraction {
          default_value {
            numerator: 100
          }
          runtime_key: "routing.traffic_shift.cluster_127_0_0_1_8877_emissary"
        }
      }
      route {
        cluster: "cluster_127_0_0_1_8877_emissary"
        prefix_rewrite: "/ambassador/v0/check_ready"
        timeout {
          seconds: 10
        }
      }
    }
    routes {
      match {
        prefix: "/ambassador/v0/check_alive"
        case_sensitive {
          value: true
        }
        headers {
          name: "x-forwarded-proto"
          exact_match: "https"
        }
        runtime_fraction {
          default_value {
            numerator: 100
          }
          runtime_key: "routing.traffic_shift.cluster_127_0_0_1_8877_emissary"
        }
      }
      route {
        cluster: "cluster_127_0_0_1_8877_emissary"
        prefix_rewrite: "/ambassador/v0/check_alive"
        timeout {
          seconds: 10
        }
      }
    }
    routes {
      match {
        prefix: "/ambassador/v0/check_alive"
        case_sensitive {
          value: true
        }
        runtime_fraction {
          default_value {
            numerator: 100
          }
          runtime_key: "routing.traffic_shift.cluster_127_0_0_1_8877_emissary"
        }
      }
      route {
        cluster: "cluster_127_0_0_1_8877_emissary"
        prefix_rewrite: "/ambassador/v0/check_alive"
        timeout {
          seconds: 10
        }
      }
    }
    routes {
      match {
        prefix: "/backend/"
        case_sensitive {
          value: true
        }
        headers {
          name: "x-forwarded-proto"
          exact_match: "https"
        }
        runtime_fraction {
          default_value {
            numerator: 100
          }
          runtime_key: "routing.traffic_shift.cluster_quote_default_default"
        }
      }
      route {
        cluster: "cluster_quote_default_default"
        prefix_rewrite: "/"
        timeout {
          seconds: 3
        }
        rate_limits {
          stage {
          }
          actions {
            request_headers {
              descriptor_key: "x-emissary-test-allow"
            }
          }
        }
      }
    }
    routes {
      match {
        prefix: "/backend/"
        case_sensitive {
          value: true
        }
        runtime_fraction {
          default_value {
            numerator: 100
          }
          runtime_key: "routing.traffic_shift.cluster_quote_default_default"
        }
      }
      route {
        cluster: "cluster_quote_default_default"
        prefix_rewrite: "/"
        timeout {
          seconds: 3
        }
        rate_limits {
          stage {
          }
          actions {
            request_headers {
              descriptor_key: "x-emissary-test-allow"
            }
          }
        }
      }
    }
  }
}
http_filters {
  name: "envoy.filters.http.cors"
}
http_filters {
  name: "envoy.filters.http.ratelimit"
  typed_config {
    [type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit] {
      domain: "emissary"
      request_type: "both"
      timeout {
        nanos: 20000000
      }
      failure_mode_deny: true
      rate_limit_service {
        grpc_service {
          envoy_grpc {
            cluster_name: "cluster_ratelimit_example_default_5000_default"
          }
        }
        transport_api_version: V3
      }
    }
  }
}
http_filters {
  name: "envoy.filters.http.router"
}
http_protocol_options {
}
server_name: "envoy"
access_log {
  name: "envoy.access_loggers.file"
  typed_config {
    [type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog] {
      path: "/dev/fd/1"
      log_format {
        text_format_source {
          inline_string: "ACCESS [%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% \"%REQ(X-FORWARDED-FOR)%\" \"%REQ(USER-AGENT)%\" \"%REQ(X-REQUEST-ID)%\" \"%REQ(:AUTHORITY)%\" \"%UPSTREAM_HOST%\"\n"
        }
      }
    }
  }
}
use_remote_address {
  value: true
}
normalize_path {
  value: true
}

(exit code 1)
Aborting update...
2024-02-28 14:32:19 diagd 3.9.1 [P23TAEW] INFO: no update performed (invalid envoy configuration generated), continuing with current configuration...
time="2024-02-28 14:32:19.4013" level=info msg="error notifying diagd: 500 INTERNAL SERVER ERROR, ignoring (invalid envoy configuration generated) in snapshot snapshot" func=github.com/emissary-ingress/emissary/v3/cmd/entrypoint.notifyWebhookUrl file="/go/cmd/entrypoint/notify.go:141" CMD=entrypoint PID=1 THREAD=/watcher/notifyCh

``
gecube commented 4 months ago

Hi! It works for me, but I implemented the rate limit with the external rate limit service. If you are interested, I can share to you the step-by-step guide.

cindymullins-dw commented 4 months ago

Emissary's rate limit example relies on an external rate limit service as noted on the page. By contrast, Edge Stack, the licensed version, comes with a rate limit service.

The RateLimitService resource tells Emissary-ingress what external service to use for rate limiting. If Emissary-ingress cannot contact the rate limit service, it will allow the request to be processed as if there were no rate limit service configuration.

grumpymatt commented 4 months ago

I had installed the example external rate limit service: https://github.com/emissary-ingress/ratelimit-example

The problem I saw was errors in the Envoy logs when it tried to reconfigure after applying rate limiting. It looked the config was bad not that it couldn't reach the rate limit service.

gecube commented 4 months ago

I used the service https://github.com/envoyproxy/ratelimit

The config for it looked like something:

---
domain: emissary
descriptors:
  - key: remote_address
    rate_limit:
      unit: second
      requests_per_unit: 1000
    shadow_mode: false
    descriptors:
      - key: generic_key
        value: api_3
        rate_limit:
          unit: second
          requests_per_unit: 30
        shadow_mode: false
      - key: generic_key
        value: api_3_order
        rate_limit:
          unit: second
          requests_per_unit: 100
        shadow_mode: false
      - key: generic_key
        value: api_v4
        rate_limit:
          unit: second
          requests_per_unit: 200
        shadow_mode: false

  - key: remote_address
    value: 10.2.132.186
    rate_limit:
      unit: second
      requests_per_unit: 0
    shadow_mode: true

  - key: remote_address
    value: 50.0.0.5
    rate_limit:
      unit: second
      requests_per_unit: 0
    shadow_mode: true

  - key: test
    value: vasya
    rate_limit:
      unit: second
      requests_per_unit: 1
    shadow_mode: true

and finally my mapping looked like:

apiVersion: getambassador.io/v3alpha1
kind: Mapping
metadata:
  name: **********
  namespace: **********
spec:
  ambassador_id: [ "public" ]
  hostname: ***********
  prefix: /api/v4
  rewrite: ""
  add_request_headers:
    Site-Enum:
      value: *********
      append: False
  service: **********
  labels:
    emissary:
      - request_label_group:
          - request_headers:
              header_name: "CF-Connecting-IP"
              key: remote_address
          - generic_key:
              key: endpoint
              value: api_v4

no errors at all