Proposal: support L4 Golang extension for tcp upstream on Envoy

duxin40 commented 3 weeks ago

One sentence explanation

Support using Golang to extend TCP upstream proxy, to make changes to connections and data messages in the http2tcp situation of envoy.

Backend

When envoy serves as an ingress gateway, we use envoy to implement http2tcp traffic, upstream services based on the TCP protocol may have development requirements such as message encapsulation, connection management, and multiplexing (such as http2dubbo).

Therefore, I am wondering if it is possible to develop a TCP upstream proxy that supports Golang extension, allowing users to extend envoy through the Go language and quickly implement message encapsulation, connection management, multiplexing, and other operations on TCP based upstream in the http2tcp situation.

Design

In Envoy's http2tcp situation, by adding the GoLang L4 extension of TCP Upstream to Envoy cluster's upstream_config , users can implement Envoy's tcp upstream extension using GoLang via the GoLang L4 upstream tcp extension SDK.

In terms of overall architecture, we refer to Envoy's existing L7 and L4 Golang extension for downstream:

C++ L4 Golang extension for tcp upstream

this is on the Envoy side implemented in C++, this module is used to call the tcp upstream extension implemented by the developer through GoLang.

GoLang L4 upstream tcp extension SDK

this exports some CGO APIs for interaction between the Envoy's C++ Upstream tcp GoLang extension and the pure GoLang upstream tcp extension.

data flow:

Here is my thought about Golang extension function points:

Support encoding message processing for upstream TCP requests (route and cluster have been determined, and targeted message processing can be performed for route and cluster)
Support the handling of conn connection status during the encoding stage of upstream TCP requests (for example, by setting end_stream=false to avoid envoy semi connected status)
Support decoding message processing and aggregation for upstream TCP response (for example, by setting end_stream=true to indicate that the message is encapsulated and can be passed to downstream)
Support upstream TCP to handle connections within onPoolReady and onPoolFailure. (For example, calling enableHalfLoss, readDisable)
Support obtaining route and cluster information, which can be referenced for targeted processing in the above stages.
Support upstream TCP connepool multiplexing, which means that multiple different requests from downstream can be processed concurrently within the same upstream TCP connection.

Golang extension configuration location:

What interfaces are implemented in c++

The c++ version of golang tcp upstream extension built into envoy needs to implement the following interface, and the user developed go version extension extends various methods. 1.1 Router::GenericConnPool -- init ConnectionPool

// Router::GenericConnPool
  virtual void newStream(GenericConnectionPoolCallbacks* callbacks) override;
  virtual bool cancelAnyPendingStream() override;

1.2 Tcp::ConnectionPool::Callbacks -- deal ConnectionPool action

 // Tcp::ConnectionPool::Callbacks
  void onPoolFailure(ConnectionPool::PoolFailureReason reason,
                     absl::string_view transport_failure_reason,
                     Upstream::HostDescriptionConstSharedPtr host) override {
    upstream_handle_ = nullptr;
    callbacks_->onPoolFailure(reason, transport_failure_reason, host);
  }

  void onPoolReady(Envoy::Tcp::ConnectionPool::ConnectionDataPtr&& conn_data,
                   Upstream::HostDescriptionConstSharedPtr host) override;

1.3 Router::GenericUpstream -- encode before send req

  // Router::GenericUpstream
  void encodeData(Buffer::Instance& data, bool end_stream) override;
  void encodeMetadata(const Envoy::Http::MetadataMapVector&) override {}
  Envoy::Http::Status encodeHeaders(const Envoy::Http::RequestHeaderMap& headers, bool end_stream) override;
  void encodeTrailers(const Envoy::Http::RequestTrailerMap&) override;
  void enableHalfClose() override;
  void readDisable(bool disable) override;
  void resetStream() override;
  void setAccount(Buffer::BufferMemoryAccountSharedPtr) override {}

1.4 Tcp::ConnectionPool::UpstreamCallbacks -- decode after recv resp

  // Tcp::ConnectionPool::UpstreamCallbacks
  void onUpstreamData(Buffer::Instance& data, bool end_stream) override;
  void onEvent(Network::ConnectionEvent event) override;
  void onAboveWriteBufferHighWatermark() override;
  void onBelowWriteBufferLowWatermark() override;
  const StreamInfo::BytesMeterSharedPtr& bytesMeter() override { return bytes_meter_; }

Best practice

Internally, in the situstion of envoy ingress gateway, we have implemented the http2dubbo function using Golang extension on envoy based on the above architecture. We believe that applying this framework to more general http2tcp situation can enable developers to quickly get started with envoy and implement http2tcp business functions~~

Looking forward to your ideas/comments on this proposal!! @doujiang24

adisuissa commented 3 weeks ago

cc @doujiang24 as codeowner of the golang filter

doujiang24 commented 3 weeks ago

@duxin40 Cool, sounds good, will take a deeper look in the next few days.

envoyproxy / envoy