Envoy rate limit v3 API fail: Unexpected HTTP/1.x request: POST /envoy.service.ratelimit.v3.RateLimitService/ShouldRateLimit

13567436138 commented 2 years ago

Issue Description

envoy rate limit fail [2021-09-06T10:24:38.050Z] "GET /productpage HTTP/1.1" 500 - rate_limiter_error - "-" 0 0 18 - "172.20.0.1" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36 Edg/93.0.961.38" "bc018697-9011-97bb-97a3-047021de0ac4" "bookinfo.com:32545" "-" outbound|9080||productpage.istio.svc.cluster.local - 172.20.0.52:8080 172.20.0.1:58561 - - [2021-09-06T10:24:39.603Z] "GET /productpage HTTP/1.1" 500 - rate_limiter_error - "-" 0 0 3 - "172.20.0.1" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36 Edg/93.0.961.38" "ff4fe770-0150-9f44-8996-e7889bd98ce2" "bookinfo.com:32545" "-" outbound|9080||productpage.istio.svc.cluster.local - 172.20.0.52:8080 172.20.0.1:58561 - - Type: bug report or feature request

Describe what happened (or what feature you want)

Describe what you expected to happen

How to reproduce it (as minimally and precisely as possible)

envoyfilters/ratelimit/sentinel/config.yaml 

kubectl apply -f config.yaml  -n istio

apiVersion: v1
kind: ConfigMap
metadata:
  name: ratelimit-config
data:
  config.yaml: |
    domain: productpage-ratelimit
    descriptors:
      - key: PATH
        value: "/productpage"
        rate_limit:
          unit: minute
          requests_per_unit: 1
      - key: PATH
        rate_limit:
          unit: minute
          requests_per_unit: 100

envoyfilters/ratelimit/sentinel/sentinel-deploy.yaml

kubectl apply -f sentinel-deploy.yaml -n istio

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sentinel-rls-server
  labels:
    app: sentinel
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sentinel
  template:
    metadata:
      labels:
        app: sentinel
    spec:
      containers:
        - name: sentinelserver
          image: "registry.cn-hangzhou.aliyuncs.com/sentinel-docker-repo/sentinel-envoy-rls-server:latest"
          imagePullPolicy: Always
          ports:
            - containerPort: 10245
            - containerPort: 8719
          volumeMounts:
            - name: sentinel-rule-config
              mountPath: /tmp/sentinel
          env:
            - name: SENTINEL_RLS_RULE_FILE_PATH
              value: "/tmp/sentinel/rule.yaml"
      volumes:
        - name: sentinel-rule-config
          configMap:
            name: ratelimit-config
            items:
              - key: config.yaml
                path: rule.yaml
---
apiVersion: v1
kind: Service
metadata:
  name: sentinel-rls-service
  labels:
    name: sentinel-rls-service
spec:
  type: ClusterIP
  ports:
    - port: 8719
      targetPort: 8719
      name: sentinel-command
    - port: 10245
      targetPort: 10245
      name: sentinel-grpc
  selector:
    app: sentinel

envoyfilters/ratelimit/sentinel/ef-sentinel-filter.yaml

kubectl apply -f ef-sentinel-filter.yaml -n istio-system

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: filter-ratelimit
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: GATEWAY
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
              subFilter:
                name: "envoy.filters.http.router"
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.ratelimit
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
            domain: productpage-ratelimit
            failure_mode_deny: true
            stage: 0
            rate_limit_service:
              grpc_service:
                envoy_grpc:
                  cluster_name: rate_limit_cluster
                timeout: 10s
              transport_api_version: V3
    - applyTo: CLUSTER
      match:
        cluster:
          service: sentinel-rls-service.istio.svc.cluster.local
      patch:
        operation: ADD
        value:
          name: rate_limit_cluster
          type: STRICT_DNS
          connect_timeout: 10s
          lb_policy: ROUND_ROBIN
          http2_protocol_options: {}
          load_assignment:
            cluster_name: rate_limit_cluster
            endpoints:
            - lb_endpoints:
              - endpoint:
                  address:
                     socket_address:
                      address: sentinel-rls-service.istio.svc.cluster.local
                      port_value: 10245

ef-sentinel-filter-action.yaml

kubectl apply -f ef-sentinel-filter-action.yaml -n istio-system

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: filter-ratelimit-svc
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
    - applyTo: VIRTUAL_HOST
      match:
        context: GATEWAY
        routeConfiguration:
          vhost:
            name: "*:80"
            route:
              action: ANY
      patch:
        operation: MERGE
        value:
          rate_limits:
          - actions: 
              - request_headers:
                  header_name: ":path"
                  descriptor_key: "PATH"

Tell us your environment

istio 1.11.0

Anything else we need to know?

sczyh30 commented 2 years ago

Hi, any error message in Sentinel RLS token server?

13567436138 commented 2 years ago

Sep 09, 2021 12:53:42 AM io.grpc.netty.NettyServerTransport notifyTerminated
INFO: Transport failed
io.netty.handler.codec.http2.Http2Exception: Unexpected HTTP/1.x request: POST /envoy.service.ratelimit.v3.RateLimitService/ShouldRateLimit 
        at io.netty.handler.codec.http2.Http2Exception.connectionError(Http2Exception.java:103)
        at io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.readClientPrefaceString(Http2ConnectionHandler.java:302)
        at io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.decode(Http2ConnectionHandler.java:239)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:438)
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:648)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:583)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:500)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:745)

13567436138 commented 2 years ago

@sczyh30 I am waiting for your solution.

winjaychan commented 2 years ago

@sczyh30 I am waiting for your solution.

which rls version you deployed?

13567436138 commented 2 years ago

registry.cn-hangzhou.aliyuncs.com/sentinel-docker-repo/sentinel-envoy-rls-server:latest

13567436138 commented 2 years ago

7天了，有人能看下么 @winjaychan

sczyh30 commented 2 years ago

If you're using v3 rate limit API, you may need to build the RLS token server jar and image by yourself. The v3 API support has not been release yet.

13567436138 commented 2 years ago

I see

13567436138 commented 2 years ago

Sep 15, 2021 12:22:59 AM io.grpc.netty.NettyServerTransport notifyTerminated
INFO: Transport failed
io.netty.handler.codec.http2.Http2Exception: Unexpected HTTP/1.x request: POST /envoy.service.ratelimit.v3.RateLimitService/ShouldRateLimit 
        at io.netty.handler.codec.http2.Http2Exception.connectionError(Http2Exception.java:103)
        at io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.readClientPrefaceString(Http2ConnectionHandler.java:302)
        at io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.decode(Http2ConnectionHandler.java:239)
        at io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:438)
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:833)

I compile the source code still have this error.

envoyfilters/ratelimit/sentinel/config.yaml 

kubectl apply -f config.yaml  -n istio

apiVersion: v1
kind: ConfigMap
metadata:
  name: ratelimit-config
data:
  config.yaml: |
    domain: productpage-ratelimit
    descriptors:
      - key: PATH
        value: "/productpage"
        rate_limit:
          unit: minute
          requests_per_unit: 1
      - key: PATH
        rate_limit:
          unit: minute
          requests_per_unit: 100

envoyfilters/ratelimit/sentinel/sentinel-deploy.yaml

kubectl apply -f sentinel-deploy.yaml -n istio

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sentinel-rls-server
  labels:
    app: sentinel
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sentinel
  template:
    metadata:
      labels:
        app: sentinel
    spec:
      containers:
        - name: sentinelserver
          image: "registry.cn-hangzhou.aliyuncs.com/hxpdocker/sentinel-envoy-rls-server:latest"
          imagePullPolicy: Always
          ports:
            - containerPort: 10245
            - containerPort: 8719
          volumeMounts:
            - name: sentinel-rule-config
              mountPath: /tmp/sentinel
          env:
            - name: SENTINEL_RLS_RULE_FILE_PATH
              value: "/tmp/sentinel/rule.yaml"
      volumes:
        - name: sentinel-rule-config
          configMap:
            name: ratelimit-config
            items:
              - key: config.yaml
                path: rule.yaml

apiVersion: v1
kind: Service
metadata:
  name: sentinel-rls-service
  labels:
    name: sentinel-rls-service
spec:
  type: ClusterIP
  ports:
    - port: 8719
      targetPort: 8719
      name: sentinel-command
    - port: 10245
      targetPort: 10245
      name: sentinel-grpc
  selector:
    app: sentinel

envoyfilters/ratelimit/sentinel/ef-sentinel-filter.yaml

kubectl apply -f ef-sentinel-filter.yaml -n istio-system

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: filter-ratelimit
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: GATEWAY
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
              subFilter:
                name: "envoy.filters.http.router"
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.ratelimit
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
            domain: productpage-ratelimit
            failure_mode_deny: true
            stage: 0
            rate_limit_service:
              grpc_service:
                envoy_grpc:
                  cluster_name: outbound|10245||sentinel-rls-service.istio.svc.cluster.local
                timeout: 10s
              transport_api_version: V3

ef-sentinel-filter-action.yaml

kubectl apply -f ef-sentinel-filter-action.yaml -n istio-system

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: filter-ratelimit-svc
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
    - applyTo: VIRTUAL_HOST
      match:
        context: GATEWAY
        routeConfiguration:
          vhost:
            name: "*:80"
            route:
              action: ANY
      patch:
        operation: MERGE
        value:
          rate_limits:
          - actions: 
              - request_headers:
                  header_name: ":path"
                  descriptor_key: "PATH"

pangsq commented 2 years ago

routeConfiguration.vhost.name does not support wildcard. https://github.com/istio/istio/pull/33690 has fixed the example envoyfilter(filter-ratelimit-svc) provided by istio.

GacuiSayweee commented 1 month ago

routeConfiguration.vhost.name does not support wildcard. istio/istio#33690 has fixed the example envoyfilter(filter-ratelimit-svc) provided by istio.

do you resolve this question ？

GacuiSayweee commented 1 month ago

my code is similar to yours , and i compile the source Sentinel- tag-1.8.7

GacuiSayweee commented 1 month ago

i can execute correctly to envoy:proxy, but i execute in envoyfilter error just like you...

alibaba / Sentinel