Feature Request: Preserve Client IP address for TCP ingress using Proxy Protocol

[marshallr12]

Is your feature request related to a problem? Please describe.
For TCP ingress, provide a way to access the client IP Address.

Describe the solution you'd like.
I need to know the client IP address for whitelisting, blacklisting, and logging client IP Addresses. Since Azure Container apps uses the Envoy proxy, preserving the client IP Address should be possible using the proxy protocol feature.

Describe alternatives you've considered.
The only alternative currently is not to use container apps and host the service inside of a VM with a public IP Address.

[JennyLawrance]

This is a very interesting ask. Implementing Proxy protocol as suggested also implies that the user will also code their app to handle the proxy header, right? I'm not clear on how many standard webservers support the proxy protocol on the server side. An alternative idea to meet your requirements will be for ACA to provide first class Logging for TCP connections. ACA already supports IP restriction feature, so the deny-list is already covered.

Will this approach meet your requirements, or am I missing something?

[duglin]

@marshallr12 for non-HTTP TCP connections, how is the client IP normally presented to your app (assuming there's some kind of proxy/middleware in the flow)?

[marshallr12]

The app is an SMTP server written in .NET using a System.Net.Socket.TcpListener which accepts connections and returns a System.Net.Socket.TcpClient. The client is then interrogated for the client IP, which is then used to trigger application-level rules to accept or reject the connection. This all works fine when hosting in a VM with a public IP. There are potentially other related apps I'd like to containerize (POP, IMAP, Mail agent).

In containers, the client IP is always an internal VNET address rather than the original external client IP. So that IP will always be internal no matter the initiating external client IP address, and would not be useful to any TCP-based application. From what I read, the proxy in ACA is Envoy, which supports the proxy protocol when configured properly and preserves the original IP (or provides the means to access it).

If you have an SMTP application which exposes a port to the public, you have to dynamically identify bad actor IP addresses for clients / spammers. This is done with a combination of firewall-like behaviors (too many connections, too many failed login attempts, etc.), but also by querying services like SpamHaus and providing the client IP to those services.

You also have to be able to identify whether the connecting MX server is authoritative for the domain associated with the sender of an email. This is done by providing the client IP address (which would belong to the connecting MX server) to a Sender Protection Framework API (SPF). Essentially, SPF uses DNS to see if the domain of the email sender corresponds to a known MX server for that domain, based on the client IP Address (i.e., the connecting MX server). SPF allows you to prevent spoofing of email senders.

A deny-list in this scenario has to be built dynamically over time like an application-layer firewall. So it is not the sort of deny list where an admin would statically configure deny-lists and white-lists.

[marshallr12]

This image shows my connection log (which uses table storage) since I've containerized my mail server. All of these connections represent spam (or hacker) attempts. However, the client address reflects the proxy, and not the original connecting address. The "App" column would be any of SMTP, SMTPS, POP, POPS, IMAP, etc. as there are multiple email-related application servers involved.

[ahmelsayed]

Yes, Today TCP apps will see the client ip as that of the last proxy rather than the actual downstream client ip. In HTTP apps, the proxy protocol is just that of injecting XFF headers which we do.

For TCP, the proxy protocol requires the server to handle a custom TCP header on the very beginning of the connection. This as @JennyLawrance mentioned requires opt-in from the application as it’s no longer just raw TCP connection.

However, since we have envoy both on edge and as a sidecar, the proxy protocol can happen between the edge proxy/loadbalancer and the sidecar helper, then the sidecar can set the correct client ip and strip that packet out keeping it transparent to the application. https://www.envoyproxy.io/docs/envoy/latest/configuration/listeners/listener_filters/original_src_filter

We don’t have this today though.

[dwgreen1]

I am also interested in preserving the client IP for TCP connections. Is there any update on whether this might be something implemented? In my case, I have an Azure Container App for which I am serving SFTP connections for clients, but it makes it impossible to do any advanced features based on client IP at the application level without it.

[matty-hall]

I have a similar use-case relating to SFTP that would benefit from this. Logging aside, it would allow for per-user IP restrictions to be implemented (i.e. user X can only login from IP Y, or, flipping it around, per-IP user restrictions).

A similar challenge I've run into is in multi-tenant scenarios in which the tenant wants to customize characteristics of the SSH channel (i.e. turn on or off ciphers, use their own public key, etc). Because of the way SSH works, there is no HTTP equivalent of a "Host" header and the SSH key exchange (which has the parameters I'd like to customize) happens prior to authentication (when I'd know who the client is). The only real option is to distinguish tenants by the public IP address they're connecting to. While this could be done by spinning up a container-app-per-tenant, it would be much more convenient if I could assign a public IP prefix to the container app and have Envoy also preserve the original destination IP.

It's unfortunate that no container offerings in Azure seem to allow for this at the TCP level rather than the HTTP level. Best I can tell, the only way to really do this would be to use Load Balancer in Floating IP mode along with a full IaaS VM, which I'd like to avoid.