help request: Layer 4 nginx is before apisix. After proxy_procto is enabled on layer 4, apisix reports 400.

[ccbutcc]

Description

route:

4层Nginx -> Apisix (IngressApisix)

为了传递真实的client Ip ，4层Nignx 配置了 proxy_protocol on;

结果apisix带理地址 直接400

此种情况在ingressNginx 上是没有开启 proxy配置导致， 求教apisix 该如何配置

Environment

• APISIX version (run apisix version):
• Operating system (run uname -a):
• OpenResty / Nginx version (run openresty -V or nginx -V):
• etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):
• APISIX Dashboard version, if relevant:
• Plugin runner version, for issues related to plugin runners:
• LuaRocks version, for installation issues (run luarocks --version):

[shreemaan-abhishek]

please update the description in english so everyone can help you, thanks.

[ccbutcc]

there is a Native Nginx front of apisix as 4 layer proxy, like as user ->Nginx 4 layer -> Apisix Ingress -> K8S . when i set the first native nginx _proxyprotocol on to send the clinet_ip to apisix , i don't know how to set the apisix ingress also support the _proxyprotocol on , and when i don't add anything on apisix ingress ,it would comes back 400 . i need the apisix ingress to get the real client ip from the first Native Nginx thanks

[shreemaan-abhishek]

https://docs.api7.ai/hub/real-ip#obtain-real-client-address-from-uri-parameter this might help you.

[ccbutcc]

thnnks, but when i i use the plugin real-ip ,it would not work well . the ip which the apisix get is the layer 4 nginx 's ip, my layer 4 nginx is as a tcp proxy

[VanLiuZhi]

I also encountered the same problem. Nginx forwards to apisix at the 4th layer. I have already configured the proxy protocol in nginx, and apisix should also be configured to support the proxy protocol. Generally, we want to read the real IP of the client. But I don't understand how to configure the proxy protocol in apisix. I found the following configuration:

#proxy_protocol:                  # Proxy Protocol configuration
#  listen_http_port: 9181         # The port with proxy protocol for http, it differs from node_listen and admin_listen.
# This port can only receive http request with proxy protocol, but node_listen & admin_listen
# can only receive http request. If you enable proxy protocol, you must use this port to
# receive http request with proxy protocol
#  listen_https_port: 9182        # The port with proxy protocol for https
#  enable_tcp_pp: true            # Enable the proxy protocol for tcp proxy, it works for stream_proxy.tcp option
#  enable_tcp_pp_to_upstream: true # Enables the proxy protocol to the upstream server

I tried to verify and found that listen_http_port must be configured. At this time, my apisix listens on port 80, so listen_http_port cannot be configured as port 80. Do I have to separate the data with proxy protocol from the data without proxy protocol?

From the comments in the configuration, can I only modify the listen, such as changing it to 8020, and then use listen_http_port as the entry point for all my traffic, which means the listen port is abandoned? Do I need to use it like this?

apisix: 
  node_listen:  
    - 8020
...
proxy_protocol:
  listen_http_port: 80
  listen_https_port: 443
  enable_tcp_pp_to_upstream: true

[shreemaan-abhishek]

@ccbutcc did you try setting the proxy_protocol configurations in the stream config here?

https://github.com/shreemaan-abhishek/apisix/blob/2d47b4b3b33730b1de92c91c4fbe2b239ee339af/conf/config-default.yaml#L212-L214

I think this should address your problem.

[shreemaan-abhishek]

Do I have to separate the data with proxy protocol from the data without proxy protocol?

@VanLiuZhi, did you try setting the same port for apisix listen and proxy protocol? Also please pay attention to: https://github.com/apache/apisix/issues/10626#issuecomment-1901741420

[adussarps]

Did anyone had success with this setup ?

I encounter the same challenge as described in this issue: https://github.com/apache/apisix/issues/11138

@VanLiuZhi Did you manage to listen to port 80 with proxy_protocol ?

[jialechan]

try to add "real_ip_from", example:

apisix:
  proxy_protocol:        
    listen_http_port: 9181        
  nginx_config:
    http:
      real_ip_header: proxy_protocol
      real_ip_from:
        - 127.0.0.1
        - "unix:"
        - 10.0.0.0/8 # your vpc CIRD

[BadTorro]

Did anyone get this up & running? I'm trying to make use of the proxy protocol with an loadbalancer and apisix (bitnami chart) deployment on Digitalocean, unfortunately ending up nowhere...

dataPlane:
  service:
    annotations:
      service.beta.kubernetes.io/do-loadbalancer-name: "lb-name"
      service.beta.kubernetes.io/do-loadbalancer-size-unit: "1"
      service.beta.kubernetes.io/do-loadbalancer-tls-passthrough: "true"
      service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: "true"
    extraConfig:
      apisix: 
        proxy_protocol: 
          listen_http_port: 9181
          listen_https_port: 9182
      nginx_config: 
        http: 
          real_ip_header: proxy_protocol

I'm getting the client IP in the data plane logs, but getting a "400 Bad request" response...(IP 1.2.3.4 in this case..)

[20/Oct/2024:11:28:08 +0000] - "PROXY TCP4 1.2.3.4 4.4.4.4 30828 80" 400 229 0.009 "-" "-" - - - "://"

---

According to this article, real-ip using the $proxy_protocol_addr should work out of the box, but did not lead to any success either...