PROXY protocol listener with `deny_unauthorized` silently closes after a non-authorized request

[bmendric]

Describe the bug A TCP listener configured with proxy_protocol_behavior = "deny_unauthorize" silently closes after receiving a request originating from an IP not listed within the proxy_protocol_authorized_addrs list.

To Reproduce Steps to reproduce the behavior:

1. Run a vault server with the configuration provided below: vault server -config=config.hcl
2. Validate that the service is running and bound to the expected port: curl http://127.0.0.1:8200/v1/sys/health / ss -tlpn | grep 8200
3. Perform a query against the Vault server using an IP address not contained within the proxy_protocol_authorized_addrs list. In my case, curl http://192.168.1.20:8200/v1/sys/health
4. Validate that the request was not serviced and the listener is no longer active: ss -tlpn | grep 8200

Expected behavior Either:

1. Anything to log an error saying that the listener closed and why
2. For the listener to reject the connection and carry on with listening

Environment:

• Vault Server Version (retrieve with vault status): Vault v1.11.3 (17250b25303c6418c283c95b1d5a9c9f16174fe8), built 2022-08-26T10:27:10Z
• Vault CLI Version (retrieve with vault version): Vault v1.11.3 (17250b25303c6418c283c95b1d5a9c9f16174fe8), built 2022-08-26T10:27:10Z
• Server Operating System/Architecture: OpenSUSE Tumbleweed x86_64

Of note, the above environment is my home lab and reflects the simple configuration testing done/described in this issue. The problem was originally discovered in a production testing environment using a three node cluster of Ubuntu 18.04 nodes running Vault 1.11.1 from within Docker containers.

Vault server configuration file(s):

storage "file" {
  path = "/tmp/vault/data"
}

listener "tcp" {
  address                         = "[::]:8200"
  tls_disable                     = true
  proxy_protocol_behavior         = "deny_unauthorized"
  proxy_protocol_authorized_addrs = "127.0.0.1"
}

log_level  = "trace"
log_format = "json"
ui         = false

disable_mlock = true

Additional context This took me quite a long time to wrap my head around due to our configuration in the production testing environment, so there are a couple interesting bits I picked up along the way.

1. This behavior does not appear with proxy_protocol_behavior set to use_always nor allow_authorized
2. The packet capture is what I expect (I think?) -- TCP handshake followed by a request and the Vault server sending a reset directly afterwards due to the lack of a PROXY protocol message

Passing --haproxy-protocol to curl resulted in the following:

3. Slightly surprising to me, the cluster port for the HA cluster is unaffected by the API port closure
4. I think there is a bug with how reject errors are handled in the proxy protocol wrapping function as the pires/go-proxyproto implementation seems to blindly close the TCP connection any time a policy resolution returns an non-nil error. I am fairly confident this isn't related, but thought it interesting

Finally, here is a screenshot of my local testing alongside the runtime logs --

[heatherezell]

Hi folks! Is this still an issue in newer versions of Vault? Please let me know so I can bubble it up accordingly. Thanks!

[bmendric]

@hsimon-hashicorp I can confirm that this is still an issue with 1.15.5. I attempted to replicate it on 1.16.1, but ran into https://github.com/hashicorp/vault/issues/26439

[heatherezell]

@hsimon-hashicorp I can confirm that this is still an issue with 1.15.5. I attempted to replicate it on 1.16.1, but ran into #26439

Thank you so much for the response! I'll keep this on our project board. I'll also check out your new issue. :)

[bmendric]

@hsimon-hashicorp looks like this was addressed as part of HCSEC-2024-14 / CVE-2024-6468

Anything that should happen w.r.t. linking this to those PRs/Issues?