search

suzuki-shunsuke / issue

MIT License
4 stars 0 forks source link

Envoy Sandbox #1

Closed suzuki-shunsuke closed 4 years ago

suzuki-shunsuke commented 5 years ago

https://www.envoyproxy.io/docs/envoy/latest/start/start#sandboxes

suzuki-shunsuke commented 5 years ago

CORS

https://www.envoyproxy.io/docs/envoy/latest/start/sandboxes/cors

image

Web UI のボタンクリック時に localhost:8000 から localhost:8002 の backend にアクセスしている。 backend の前に Envoy を立て、CORS フィルタリングしている。

open

100% 許可している

cors:
  allow_origin:
  - "*"
  allow_methods: "GET"
  filter_enabled:
    default_value:
      numerator: 100
      denominator: HUNDRED
    runtime_key: cors.www.enabled

disbled

0/100 許可している(つまり許可していない)

cors:
  filter_enabled:
    default_value:
      numerator: 0
      denominator: HUNDRED

restricted

cors:
  allow_origin:
  - "envoyproxy.io"
  allow_methods: "GET"

envoyproxy.io からの GET リクエストのみ許可している

suzuki-shunsuke commented 5 years ago

CSRF

https://www.envoyproxy.io/docs/envoy/latest/start/sandboxes/csrf

default

デフォルトだと 異なる origin からのリクエストを 100/100 拒否する

per_filter_config:
  envoy.csrf:
    filter_enabled:
      default_value:
        numerator: 100
        denominator: HUNDRED
      runtime_key: csrf.www.enabled
    shadow_enabled:
      default_value:
        numerator: 0
        denominator: HUNDRED
      runtime_key: csrf.www.shadow_enabled

disabled

envoy.csrf:
  filter_enabled:
    default_value:
      numerator: 0
      denominator: HUNDRED

shadow

additional_origin

suzuki-shunsuke commented 5 years ago

Front Proxy

https://www.envoyproxy.io/docs/envoy/latest/start/sandboxes/front_proxy

Envoy で ロードバランシングしている。

- name: service1
  connect_timeout: 0.25s
  type: strict_dns
  lb_policy: round_robin
  http2_protocol_options: {}
  load_assignment:
    cluster_name: service1
    endpoints:
    - lb_endpoints:
      - endpoint:
          address:
            socket_address:
              address: service1
              port_value: 80

strict dns

suzuki-shunsuke commented 5 years ago

gPRC bridge

https://www.envoyproxy.io/docs/envoy/latest/start/sandboxes/grpc_bridge

image

Client は gRPC ではなく、 HTTP/1 でリクエストしている。 Envoy の gRPC HTTP/1.1 bridge filter によって HTTP/2 gRPC に変換している。

Header の content-type は application/grpc でないといけない。 リクエストのパスが gRPC のメソッド名になる。

body も gPRC にシリアライズされる。

The body should be the serialized grpc body which is:

* 1 byte of zero (not compressed).
* network order 4 bytes of proto message length.
* serialized proto message.

The Python client makes HTTP/1 requests through the Envoy sidecar process which are upgraded into HTTP/2 gRPC requests. Response trailers are then buffered and sent back to the client as a HTTP/1 header payload.

https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/grpc_http1_bridge_filter

client sidecar envoy

http_filters:
- name: envoy.grpc_http1_bridge
  typed_config: {}

server sidecar envoy

routes:
- match:
  prefix: "/"
  grpc: {}

https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/route/route.proto#envoy-api-field-route-routematch-grpc

grpc
(route.RouteMatch.GrpcRouteMatchOptions)
If specified, only gRPC requests will be matched.
suzuki-shunsuke commented 5 years ago

Jaeger Native tracing

https://www.envoyproxy.io/docs/envoy/latest/start/sandboxes/jaeger_native_tracing

image

Jaeger にデータを送る設定

tracing:
  http:
    name: envoy.dynamic.ot
    typed_config:
      "@type": type.googleapis.com/envoy.config.trace.v2.DynamicOtConfig
      library: /usr/local/lib/libjaegertracing_plugin.so
      config:
        service_name: front-proxy
        sampler:
          type: const
          param: 1
        reporter:
          localAgentHostPort: jaeger:6831
        headers:
          jaegerDebugHeader: jaeger-debug-id
          jaegerBaggageHeader: jaeger-baggage
          traceBaggageHeaderPrefix: uberctx-
        baggage_restrictions:
          denyBaggageOnInitializationFailure: false
          hostPort: ""

image

ここで気になるのが、 service1 の tracing と service2 の tracing をいかに関連付けているか

service1 (Flask) から service2 にリクエストを送る際に、 service1 の request header を service2 へのリクエストにセットしている。

https://github.com/envoyproxy/envoy/blob/4e858f17fe08224c9c089240908ccd0c518e01a7/examples/front-proxy/service.py

One of the most important benefits of tracing from Envoy is that it will take care of propagating the traces to the Jaeger service cluster. However, in order to fully take advantage of tracing, the application has to propagate trace headers that Envoy generates, while making calls to other services. In the sandbox we have provided, the simple flask app (see trace function in /examples/front-proxy/service.py) acting as service1 propagates the trace headers while making an outbound call to service2.

suzuki-shunsuke commented 5 years ago

MySQL

MySQL へのクエリを Envoy で Proxy している。 MySQL 固有の stats を取得することができる。

https://github.com/envoyproxy/envoy/blob/4e858f17fe08224c9c089240908ccd0c518e01a7/examples/mysql/envoy.yaml#L10-L18

- name: envoy.filters.network.mysql_proxy
  typed_config:
    "@type": type.googleapis.com/envoy.config.filter.network.mysql_proxy.v1alpha1.MySQLProxy
    stat_prefix: egress_mysql
- name: envoy.tcp_proxy
  typed_config:
    "@type": type.googleapis.com/envoy.config.filter.network.tcp_proxy.v2.TcpProxy
    stat_prefix: mysql_tcp
    cluster: mysql_cluster
suzuki-shunsuke commented 5 years ago

traffic splitting

https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_conn_man/traffic_splitting.html

weighted_clusters を使えば、 canary release や A/B test ができそう

https://github.com/suzuki-shunsuke/issue/issues/11

suzuki-shunsuke commented 5 years ago

TLS

https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/security/ssl.html

suzuki-shunsuke commented 5 years ago

Datadog tracing

suzuki-shunsuke commented 5 years ago

Circuit breaker

suzuki-shunsuke commented 5 years ago

Retry policy

suzuki-shunsuke commented 5 years ago

envoy.yaml 書いてみて envoy を起動しようとしたところ、起動に失敗。 エラーを確認するためにローカルで起動してみる。 Docker image は配布されているもののバイナリは配布されていなさそう。

$ docker run -ti --rm -v $PWD:/workspace -w /workspace --entrypoint bash envoyproxy/envoy:v1.11.1
# envoy -c base/envoy.yaml

エラー吐いて起動に失敗

error initializing configuration 'base/envoy.yaml': Proto constraint validation failed (BootstrapValidationError.StaticResources: ["embedded message failed validation"] | caused by StaticResourcesValidationError.Clusters[i]: ["embedded message failed validation"] | caused by ClusterValidationError.LoadAssignment: ["embedded message failed validation"] | caused by ClusterLoadAssignmentValidationError.Endpoints[i]: ["embedded message failed validation"] | caused by LocalityLbEndpointsValidationError.LbEndpoints[i]: ["embedded message failed validation"] | caused by LbEndpointValidationError.Endpoint: ["embedded message failed validation"] | caused by EndpointValidationError.Address: ["embedded message failed validation"] | caused by AddressValidationError.SocketAddress: ["embedded message failed validation"] | caused by field: "port_specifier", reason: is required)