Support for Request MetadataKind in InternalUpstreamTransport

[deveshkandpal1224]

Title: Support for Request MetadataKind in InternalUpstreamTransport

Description:

I have a usecase where I am extracting an http header in a Lua filter, setting it in dynamicMetadata and passing it to an internal listener which has a TCPProxy filter configured with tunneling_config. The value that is set in dynamicMetadata needs to be referenced in the tunneling_config.

It seems there is support for passthrough_metadata for passing information from one listener to the other listener in this case. However, even though the docs seem to suggest that request metadataKind is supported ( which is needed for passing DynamicMetadata ), it seems the internal implementation is missing here and here

Is my understanding correct ? Happy to work on a PR if I can get any pointers!

[optional Relevant Links:]

Any extra documentation required to understand the issue.

[ramaraochavali]

@kyessenov Can you please take a look and suggest the path forward here?

[kyessenov]

Yeah, the assessment is correct. I couldn't make it work with the request metadata because upstream is designed to handle multiplexing, sharing multiple streams from downstream to one stream from upstream. This is done via transport socket options, which are fairly lightweight and miss a lot of info from downstream, and the missing info includes the dynamic metadata.

The most effective way is to create a filter state object instead of the dynamic metadata that is explicitly marked as shared with upstream, and which is then propagated to the transport socket options. You can use the freshly added https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/set_filter_state, for example.

[marc-barry]

I'm trying to do the same. In the docs under https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/network/tcp_proxy/v3/tcp_proxy.proto#extensions-filters-network-tcp-proxy-v3-tcpproxy-tunnelingconfig they give an example of the following:

tunneling_config:
  hostname: "%DYNAMIC_METADATA(tunnel:address)%"

Does this mean I should use https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/set_filter_state to set an object_key of tunnel.address?

[marc-barry]

Does this mean I should use https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/set_filter_state to set an object_key of tunnel.address?

I'm guessing that is wrong as I get the following error:

{"level":"info","ts":1717095853.4897122,"msg":"[2024-05-30 15:04:13.489][7538027][warning][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:138] gRPC config for type.googleapis.com/envoy.config.listener.v3.Listener rejected: Error adding/updating listener(s) tcp_0.0.0.0:16080: 'tunnel.address' does not have an object factory"}

All I'm trying to do is save %REQ(:AUTHORITY)% for use in an internal listener.

[marc-barry]

I created https://github.com/envoyproxy/envoy/issues/34435 as I think my needs might be a little different. @deveshkandpal1224 if you have any advice please let me know.