alibaba / higress

🤖 AI Gateway | AI Native API Gateway
https://higress.io
Apache License 2.0
3.52k stars 525 forks source link

使用基于Key集群限流插件报错 #1486

Closed bianshuangwei closed 1 week ago

bianshuangwei commented 2 weeks ago

Higress版本:2.0.1 部署方式:单机部署 部署服务器:centos7 基于Key集群限流插件配置详情: redis: password: "capitek" service_name: "172.18.2.27" service_port: 6379

username: "capitek"

rule_items:

redis情况说明:单节点redis,可以正常使用。配置的redis的service name为redis的ip地址,还尝试在域名中配置了此ip的dns,调用短的host添加了ip和域名的映射,依然不可用,报错信息如下: gateway-1 | 2024-11-07T07:57:44.114044Z info wasm fetching image plugins/cluster-key-rate-limit from registry higress-registry.cn-hangzhou.cr.aliyuncs.com with tag latest gateway-1 | 2024-11-07T07:57:46.849504Z critical envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1396 wasm log: failed to init redis: error status returned by host: bad argument thread=25 gateway-1 | 2024-11-07T07:57:46.849626Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1390 wasm log: [cluster-key-rate-limit] parse rule config failed: error status returned by host: bad argument thread=25 gateway-1 | 2024-11-07T07:57:46.849637Z error envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:151 Wasm VM failed Failed to configure base Wasm plugin thread=25 gateway-1 | 2024-11-07T07:57:46.851424Z critical envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:547 Plugin configured to fail closed failed to load thread=25 gateway-1 | 2024-11-07T07:57:46.851439Z critical envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:547 Plugin configured to fail closed failed to load thread=34 gateway-1 | 2024-11-07T07:57:46.851479Z critical envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:547 Plugin configured to fail closed failed to load thread=35 gateway-1 | 2024-11-07T07:57:46.851480Z critical envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:547 Plugin configured to fail closed failed to load thread=33 gateway-1 | 2024-11-07T07:57:46.851505Z critical envoy wasm external/envoy/source/extensions/common/wasm/wasm.cc:547 Plugin configured to fail closed failed to load thread=32

发生此问题后,有一个致命问题:开启此插件配置之前,路由调用成功,配置出现上述错误后,所有路由(不止配置插件的路由)调用都会失败,删除重新添加路由依然调用失败,重启higress(shutdown.sh后startup.sh)路由依然调用依然返回500错误,只有重新configure(清空重新配置启动)后重新配置路由才可调用。这个是个致命问题,一个插件的配置错误,导致整个higress的路由不可访问。

johnlanni commented 2 weeks ago

需要给wasmplugin的 failStrategy设置为 FAIL_OPEN,这样插件配置出错就会 fail over,新版本控制台下发已经做了调整,建议你先手动配置下,参考 istio 的文档说明: https://istio.io/latest/docs/reference/config/proxy_extensions/wasm-plugin/#FailStrategy

johnlanni commented 2 weeks ago

parse rule config failed: error status returned by host: bad argument

这个报错就是服务发现不了,redis 的 service name 配的不对,可以看下这个插件的文档image

bianshuangwei commented 2 weeks ago

parse rule config failed: error status returned by host: bad argument

这个报错就是服务发现不了,redis 的 service name 配的不对,可以看下这个插件的文档image

你好,我测试环境的服务器没有域名,只有ip。redis是直接部署在虚拟机上的,不是K8S或者docker部署的,redis主机的hostname是capitek100,在higress的域名配置里配置了域名prd100.cn,在服务器的/etc/hosts里做了ip和域名的映射。我将redis.service_name修改为:capitek100.prd100.cn 结果测试还是不行。路由对应的微服务直接部署在虚拟机上,非容器化部署。这种情况有办法配置redis的service name吗?

johnlanni commented 2 weeks ago

@bianshuangwei 你的服务是通过控制台加上的吗,控制台的服务列表里的服务名字就是这个service_name

bianshuangwei commented 2 weeks ago

@bianshuangwei 你的服务是通过控制台加上的吗,控制台的服务列表里的服务名字就是这个service_name

服务是通过控制台加上去的,配置的是nacos2.0作为服务来源。redis的服务通过配置固定地址,将redis注册到服务列表里,redis的服务名称显示是:redis.static,在路由的策略“基于Key集群限流”中redis的service_name设置为redis.static。配置如下:

redis: password: "capitek" service_name: "redis.static" service_port: 6379

这样配置后还是报错: gateway-1 | 2024-11-08T07:07:50.717368Z critical envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1396 wasm log: failed to init redis: error status returned by host: bad argument thread=23 gateway-1 | 2024-11-08T07:07:50.717403Z warning envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1390 wasm log: [cluster-key-rate-limit] parse rule config failed: error status returned by host: bad argument thread=23

johnlanni commented 2 weeks ago

@bianshuangwei static service目前用的逻辑端口,service_port不用填,或者填80