Envoy doesn't work anymore because of deprecated cors policy allowing

[Emixam23]

Hey!

Today my envoy proxy stopped working after I rebuild my Docker image

I am getting that error using your example:

[2020-01-19 03:38:58.528][6][critical][main] [source/server/server.cc:94] error initializing configuration '/etc/envoy.yaml': Proto constraint validation failed (Using deprecated option 'envoy.api.v2.route.CorsPolicy.allow_origin' from file route_components.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/intro/deprecated for details. If continued use of this field is absolutely necessary, see https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime#using-runtime-overrides-for-deprecated-features for how to apply a temporary and highly discouraged override.): allow_origin: "*"
allow_methods: "GET, PUT, DELETE, POST, OPTIONS"
allow_headers: "keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout"
expose_headers: "custom-header-1,grpc-status,grpc-message"
max_age: "1728000"

Dockerfile

# This configuration will build a Docker container containing
# an Envoy proxy that routes to Google.

FROM envoyproxy/envoy-dev:latest
RUN apt-get update
COPY envoy.yaml /etc/envoy.yaml
CMD /usr/local/bin/envoy -c /etc/envoy.yaml

[MalcolmJRosse]

This fix the issue

[Emixam23]

Hey,

Thanks, I will try and come back to you by tomorrow

[swuecho]

well, probably not the best option, but you do not have to use latest envoy version. To me, envoy change too much across versions.

[Emixam23]

But then, which one should I choose? Because it doesn't seem like I the latest one

[MalcolmJRosse]

you must add those 2 red blocks on your yaml file

[Emixam23]

I tried the red blocks, which gives:

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address: { address: 0.0.0.0, port_value: 9901 }

static_resources:
  listeners:
    - name: listener_0
      address:
        socket_address: { address: 0.0.0.0, port_value: 9000 }
      filter_chains:
        - filters:
            - name: envoy.http_connection_manager
              config:
                codec_type: auto
                stat_prefix: ingress_http
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: local_service
                      domains: ["*"]
                      routes:
                        - match: { prefix: "/" }
                          route:
                            cluster: api_interface
                            max_grpc_timeout: 2s
                            cors:
                              allow_origin_string_match:
                              - safe_regex:
                                  google_re2: {}
                                  regex: \*
                              allow_methods: GET, PUT, DELETE, POST, OPTIONS
                              allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
                              max_age: "1728000"
                              expose_headers: custom-header-1,grpc-status,grpc-message
                              filter_enabled:
                                default_value: {numerator: 100, denominator: HUNDRED}
                                runtime_key: cors.www.enabled
  clusters:
    - name: api_interface
      connect_timeout: 0.25s
      type: logical_dns
      http2_protocol_options: {}
      lb_policy: round_robin
      hosts: [{ socket_address: { address: host.docker.internal, port_value: 10000 }}]

I can't really tell what is happening, the client (ReactJs) pings but... nothing is ever returned. The server doesn't even get reached (Go)

MacBook-Pro-de-Emixam23:api emixam23$ docker logs envoy
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:251] initializing epoch 0 (hot restart version=11.104)
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:253] statically linked extensions:
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.grpc_credentials: envoy.grpc_credentials.aws_iam, envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.previous_hosts
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.filters.udp_listener: envoy.filters.udp_listener.udp_proxy
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.health_checkers: envoy.health_checkers.redis
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.tap, envoy.transport_sockets.tls, raw_buffer, tls
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.dubbo_proxy.route_matchers: default
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.dubbo_proxy.protocols: dubbo
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.tap, envoy.transport_sockets.tls, raw_buffer, tls
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.thrift_proxy.transports: auto, framed, header, unframed
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.filters.http: envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.fault, envoy.filters.http.adaptive_concurrency, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_stats, envoy.filters.http.header_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.rbac, envoy.filters.http.tap, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.gzip, envoy.health_check, envoy.http_dynamo_filter, envoy.ip_tagging, envoy.lua, envoy.rate_limit, envoy.router, envoy.squash
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.resource_monitors: envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.clusters: envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.dynamic_forward_proxy, envoy.clusters.redis
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.dubbo_proxy.serializers: dubbo.hessian2
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.stats_sinks: envoy.dog_statsd, envoy.metrics_service, envoy.stat_sinks.hystrix, envoy.statsd
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.dubbo_proxy.filters: envoy.filters.dubbo.router
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.udp_listeners: raw_udp_listener
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.thrift_proxy.filters: envoy.filters.thrift.rate_limit, envoy.filters.thrift.router
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.tracers: envoy.dynamic.ot, envoy.lightstep, envoy.tracers.datadog, envoy.tracers.opencensus, envoy.tracers.xray, envoy.zipkin
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.resolvers: envoy.ip
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.access_loggers: envoy.file_access_log, envoy.http_grpc_access_log, envoy.tcp_grpc_access_log
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.filters.network: envoy.client_ssl_auth, envoy.echo, envoy.ext_authz, envoy.filters.network.dubbo_proxy, envoy.filters.network.kafka_broker, envoy.filters.network.local_ratelimit, envoy.filters.network.mysql_proxy, envoy.filters.network.rbac, envoy.filters.network.sni_cluster, envoy.filters.network.thrift_proxy, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.filters.listener: envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector
[2020-01-20 15:11:13.258][7][info][main] [source/server/server.cc:255]   envoy.retry_priorities: envoy.retry_priorities.previous_priorities
[2020-01-20 15:11:13.264][7][warning][misc] [source/common/protobuf/utility.cc:441] Using deprecated option 'envoy.api.v2.listener.Filter.config' from file listener_components.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/intro/deprecated for details.
[2020-01-20 15:11:13.264][7][warning][misc] [source/common/protobuf/utility.cc:441] Using deprecated option 'envoy.api.v2.Cluster.hosts' from file cluster.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/intro/deprecated for details.
[2020-01-20 15:11:13.265][7][info][main] [source/server/server.cc:336] admin address: 0.0.0.0:9901
[2020-01-20 15:11:13.266][7][info][main] [source/server/server.cc:455] runtime: layers:
  - name: base
    static_layer:
      {}
  - name: admin
    admin_layer:
      {}
[2020-01-20 15:11:13.266][7][info][config] [source/server/configuration_impl.cc:62] loading 0 static secret(s)
[2020-01-20 15:11:13.266][7][info][config] [source/server/configuration_impl.cc:68] loading 1 cluster(s)
[2020-01-20 15:11:13.268][7][info][config] [source/server/configuration_impl.cc:72] loading 1 listener(s)
[2020-01-20 15:11:13.271][7][info][config] [source/server/configuration_impl.cc:97] loading tracing configuration
[2020-01-20 15:11:13.271][7][info][config] [source/server/configuration_impl.cc:116] loading stats sink configuration
[2020-01-20 15:11:13.271][7][info][main] [source/server/server.cc:550] starting main dispatch loop
[2020-01-20 15:11:13.272][7][info][upstream] [source/common/upstream/cluster_manager_impl.cc:171] cm init: all clusters initialized
[2020-01-20 15:11:13.272][7][info][main] [source/server/server.cc:529] all clusters initialized. initializing init manager
[2020-01-20 15:11:13.272][7][info][config] [source/server/listener_manager_impl.cc:707] all dependencies initialized. starting workers
[2020-01-20 15:26:12.194][7][info][main] [source/server/drain_manager_impl.cc:68] shutting down parent after drain

[stanley-cheung]

Pinning Envoy back to FROM envoyproxy/envoy:v1.12.2 seems to have fixed the issue. Still looking into this.

[Emixam23]

After trying, again and again, different approaches, I finally have this yaml file and it works. However, I don't understand why.. I just tried random stuff x)

static_resources:
  listeners:
    - name: listener_0
      address:
        socket_address: { address: 0.0.0.0, port_value: 9000 }
      filter_chains:
        - filters:
            - name: envoy.http_connection_manager
              config:
                codec_type: auto
                stat_prefix: ingress_http
                route_config:
                  name: local_route
                  virtual_hosts:
                  - name: local_service
                    domains: ["*"]
                    routes:
                    - match:
                        prefix: "/"
                      route:
                        cluster: api_interface
                        max_grpc_timeout: 2s
                    cors:
                      allow_origin_string_match:
                        - safe_regex:
                            google_re2: {}
                            regex: \*
                      allow_methods: GET, PUT, DELETE, POST, OPTIONS
                      allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
                      max_age: "1728000"
                      expose_headers: custom-header-1,grpc-status,grpc-message
                http_filters:
                  - name: envoy.grpc_web
                  - name: envoy.cors
                  - name: envoy.router
  clusters:
    - name: api_interface
      connect_timeout: 0.25s
      type: logical_dns
      http2_protocol_options: {}
      lb_policy: round_robin
      hosts: [{ socket_address: { address: host.docker.internal, port_value: 10000 }}]

[maurodelazeri]

im using FROM envoyproxy/envoy:v1.12.2 for now, works fine... anything above doesn't

[EdwinBetanc0urt]

I tested it and it works fine for me with and without the lines:

                filter_enabled:
                  default_value: {
                    numerator: 100,
                    denominator: HUNDRED
                  }
                  runtime_key: cors.www.enabled

Yet he throws the warning at me:

[2020-02-12 20:27:02.416][6][warning][misc] [source/common/protobuf/utility.cc:441] Using deprecated option 'envoy.api.v2.listener.Filter.config' from file listener_components.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/intro/deprecated for details.,

[kmturley]

Using deprecated option 'envoy.api.v2.route.CorsPolicy.allow_origin'

It says in the documentation:

allow_origin and allow_origin_regex have been deprecated in favor of allow_origin_string_match.

So I changed to:

allow_origin_string_match:
  - prefix: "*"

And then for this error:

Using deprecated option 'envoy.api.v2.route.CorsPolicy.enabled'

It says in the documentation:

enabled is deprecated. Set the filter_enabled field instead.

I just removed the enabled attribute since it's on by default!

enabled: true

[ramsrib]

Since grpc-web only supports POST method and the browser only uses OPTIONS for preflight request. I guess allowing only those two methods is good enough for envoy. Isn't it?

allow_methods: POST, OPTIONS