alibaba / higress

🤖 AI Gateway | AI Native API Gateway
https://higress.io
Apache License 2.0
3.22k stars 509 forks source link

Higress-gateway forward the request to the ip address which is not exist in cluster. #1165

Closed haifzhu closed 2 months ago

haifzhu commented 3 months ago

If you are reporting any crash or any potential security issue, do not open an issue in this repo. Please report the issue via ASRC(Alibaba Security Response Center) where the issue will be triaged appropriately.

Ⅰ. Issue Description

One higress-gateway instance(pod) forward the request to the ip address which is not exist in cluster。

Spec:
  Http:
    Match:
      Uri:
        Prefix:  /api/biz-eco/isv/zjzwdata
    Route:
      Destination:
        Host:  web-isv-zjzwdata
        Port:
          Number:  8080

image

image

the request match the right route rule, but the upstream_host is not exact the right one(10.11.16.173:8080) but is (10.11.27.185:8080)

Another higress-gateway instance is forward right ip address.

image

Ⅱ. Describe what happened

higress-gateway has warning logs dump.

If there is an exception, please attach the exception trace:

[Envoy (Epoch 0)] [2024-07-31 03:24:42.398][41][warning][client] [C42155999] Connection is closed by peer during connecting.
[Envoy (Epoch 0)] [2024-07-31 03:24:42.421][41][warning][client] [C42156000] Connection is closed by peer during connecting.
[Envoy (Epoch 0)] [2024-07-31 03:24:42.468][41][warning][client] [C42156001] Connection is closed by peer during connecting.

Ⅲ. Describe what you expected to happen

Ⅳ. How to reproduce it (as minimally and precisely as possible)

  1. xxx
  2. xxx
  3. xxx

Ⅴ. Anything else we need to know?

Ⅵ. Environment:

johnlanni commented 3 months ago

https://github.com/alibaba/higress/issues/1129#issuecomment-2257273945

应该是这个问题导致的,近期会backport一下

p.s. 标准按照配置配了 PILOT_XDS_SEND_TIMEOUT 变量,不会遇到一直阻塞的问题

haifzhu commented 3 months ago

好,目前已经在环境配置了PILOT_XDS_SEND_TIMEOUT变量

johnlanni commented 2 months ago

已经发布了 2.0.0-rc.1 修复了此问题