cloudwego / kitex

Go RPC framework with high-performance and strong-extensibility for building micro-services.
https://www.cloudwego.io
Apache License 2.0
6.95k stars 807 forks source link

[XDS] client, receive failed, error=connection error: desc = "transport is closing" #875

Open booldesign opened 1 year ago

booldesign commented 1 year ago

测试环境:istio,服务发现是用原生的istio 系统版本: k8s:版本:v1.25.4 istio版本:1.17.1 docker 20.10.23

遇到的问题: proxyless模式,pod运行一会后一直报 2023/03/20 08:11:43.723511 client.go:227: [Error] KITEX: [XDS] client, receive failed, error=connection error: desc = "transport is closing" 2023/03/20 08:42:20.372792 client.go:227: [Error] KITEX: [XDS] client, receive failed, error=connection error: desc = "transport is closing" 2023/03/20 09:15:12.557724 client.go:227: [Error] KITEX: [XDS] client, receive failed, error=connection error: desc = "transport is closing"

ppzqh commented 1 year ago

一直报?看你贴出来的日志是半小时左右一次。 可以看一下istiod的日志,看看是否有错误日志。

booldesign commented 1 year ago

只有几个warn,其它都ads的info 2023-03-20T09:21:56.030703Z warn model Istio Version is not found in metadata for api-useraccount-v1-6475fd8c4b-bltjl.mesh-cloud, which may have undesirable side effects

booldesign commented 1 year ago

第一请求正确响应,第二次请求就500,过段时间再请求,又正确响应了,pod没有重启 500报如下错误 [Error] KITEX: processing request error, remoteService=, remoteAddr=10.9.0.18:50442, error=biz error: no matched route for service code.cloud.svc.cluster.local:8080, err=get route failed: [XDS] manager, fetch RouteConfig resource[code.cloud.svc.cluster.local:8080] timeout

ppzqh commented 1 year ago

能提供一下你请求的代码吗?

booldesign commented 1 year ago
func init() {
    if err := xdsmanager.Init(xdsmanager.WithXDSServerAddress("istiod.istio-system.svc:15010")); err != nil {
        panic(err)
    }
}

func InitVCodeClient() {
    var (
        destService string
        options     []client.Option
    )
    destService, options = k8sVCodeOption()

    var err error
    VCodeClient, err = vcodesrv.NewClient(
        destService,
        options...,
    )
    if err != nil {
        panic(err)
    }
}

func k8sVCodeOption() (string, []client.Option) {
    return "code.cloud.svc.cluster.local:8080", []client.Option{
        client.WithSuite(tracing.NewClientSuite()), // tracer
        client.WithXDSSuite(xds.ClientSuite{
            RouterMiddleware: xdssuite.NewXDSRouterMiddleware(
                xdssuite.WithRouterMetaExtractor(metadata.ExtractFromPropagator),
            ),
            Resolver: xdssuite.NewXDSResolver(),
        }),
    }
}
ppzqh commented 1 year ago

请问是否有配置destinationRule?

SpecialYang commented 1 year ago

istio server设置了默认连接空闲时间是30分钟,如果这段时间内没有xds推送,那么连接就会被istio关闭了。

https://github.com/istio/istio/pull/10870

ppzqh commented 1 year ago

感谢提供信息

tankpanv commented 1 year ago

第一请求正确响应,第二次请求就500,过段时间再请求,又正确响应了,pod没有重启 500报如下错误 [Error] KITEX: processing request error, remoteService=, remoteAddr=10.9.0.18:50442, error=biz error: no matched route for service code.cloud.svc.cluster.local:8080, err=get route failed: [XDS] manager, fetch RouteConfig resource[code.cloud.svc.cluster.local:8080] timeout

你好,这个问题最终是怎么解决的呢,是需要配置哪个配置参数?

ppzqh commented 1 year ago

第一请求正确响应,第二次请求就500,过段时间再请求,又正确响应了,pod没有重启 500报如下错误 [Error] KITEX: processing request error, remoteService=, remoteAddr=10.9.0.18:50442, error=biz error: no matched route for service code.cloud.svc.cluster.local:8080, err=get route failed: [XDS] manager, fetch RouteConfig resource[code.cloud.svc.cluster.local:8080] timeout

你好,这个问题最终是怎么解决的呢,是需要配置哪个配置参数?

错误日志也是半小时一次吗?