search

linkerd / linkerd2

Ultralight, security-first service mesh for Kubernetes. Main repo for Linkerd 2.x.
https://linkerd.io
Apache License 2.0
10.63k stars 1.28k forks source link

CLI: disable protocol detection instead of skipping ports #1285

Closed klingerf closed 5 years ago

klingerf commented 6 years ago

The --skip-inbound-ports and --skip-outboud-ports flags passed to linkerd inject are used by the proxy-init container to modify the list of ignored ports in the pod's iptables rules. This causes requests to those ports to bypass the proxy altogether.

At this point, the primary (only?) reason to use those flags is to accommodate server-speaks-first protocols on non-default ports, for which the proxy cannot reliably determine the protocol. It's possible to configure the proxy to disable protocol detection on specific ports, however, using the following variables:

// These *disable* our protocol detection for connections whose SO_ORIGINAL_DST
// has a port in the provided list.
pub const ENV_INBOUND_PORTS_DISABLE_PROTOCOL_DETECTION: &str =
    "LINKERD2_PROXY_INBOUND_PORTS_DISABLE_PROTOCOL_DETECTION";
pub const ENV_OUTBOUND_PORTS_DISABLE_PROTOCOL_DETECTION: &str =
    "LINKERD2_PROXY_OUTBOUND_PORTS_DISABLE_PROTOCOL_DETECTION";

So it seems feasible that, instead of bypassing the proxy altogether, we could just disable protocol detection, which would result in the proxy being able to proxy these requests as TCP.

I'm on the fence as to whether or not we'd want to repurpose the existing CLI flags or create new ones for this.

Relates to #1281.

briansmith commented 6 years ago

the primary (only?) reason

I can think of a few other reasons one might want to skip the proxy completely:

  1. The application being proxied depends on having access to the real remote address. When the proxy is proxying, I'm not sure if the application sees the proxy's IP address as the remote or if it sees the same remote address it would see if the proxy isn't present.
  2. Performance. The proxy has some cost and some applications may not want to risk paying any cost.
  3. A/B comparison. The user may want to compare the performance and/or behavior of the application with Conduit injected but without proxying on a particular port, e.g. for troubleshooting.

I'm on the fence as to whether or not we'd want to repurpose the existing CLI flags or create new ones for this.

I think users would expect flags named like this to completely bypass the proxy on those ports, so if we were to remove the full "skip" functionality then we should try to find better names if we can come up with any.

owenhaynes commented 5 years ago

Something similar is needed to talk to external cluster file servers. At the moment its not possible to use ceph-fuse inside of a pod as the required ports are in a range 6800-7300 + some others. So it seems to be better to just skip the proxy by ip?

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

grampelberg commented 5 years ago

@olix0r is this still valid? Seems pretty useful.

olix0r commented 5 years ago

As far as I understand, the skip-ports configurations impact both iptables and the proxy today. So, it's probably technically possible to stop setting this configuration on proxy-init, though I'm not clear on the benefits of using the proxy in this scenario.

grampelberg commented 5 years ago

TCP metrics for now, possibility of TLS in the future (obviously with lots of details to sort out).

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

007 commented 5 years ago

@grampelberg or @olix0r reopen as per the Feb 27th comment?

I'm looking for a ticket to track TLS for TCP, and this seems as good as any others, please redirect me if I'm missing a feature request ticket that I should follow instead.

grampelberg commented 5 years ago

Would you mind opening a new issue that is specific to your use case @007? It is super helpful for scoping to have something that is exactly what you're looking for. Getting details about what protocols you're looking to have mTLS'd, where this all exists in your infrastructure and how they talk together helps make sure the solution is exactly what you want.

007 commented 5 years ago

Tks @grampelberg, #3207 filed.