search

apache / dubbo

The java implementation of Apache Dubbo. An RPC and microservice framework.
https://dubbo.apache.org/
Apache License 2.0
40.41k stars 26.42k forks source link

"dubbo.application.qosAcceptForeignIp=true" doesn't work in a proxyless mesh microservice which is not only a consumer but also a provider. #10592

Closed zewade closed 2 years ago

zewade commented 2 years ago

Environment

Steps to reproduce this issue

  1. Checkout apache/dubbo-samples projects from github，and open dubbo-samples-xds-consumer sub project.

  2. Add a demo provider class to the project, which uses triple protocol. for example:

    
package org.apache.dubbo.samples.api;

public interface SimpleMathService {

int add(int a, int b);

}

package org.apache.dubbo.samples.action;

import org.apache.dubbo.config.annotation.DubboService; import org.apache.dubbo.samples.api.SimpleMathService;

@DubboService(version = "1.0.0") public class SimpleMathServiceImpl implements SimpleMathService { @Override public int add(int a, int b) { return 0; } }


3. Add `dubbo.protocol.name=tri` to `classpath:/spring/dubbo-consumer.properties`.

dubbo.application.name=dubbo-samples-xds-consumer dubbo.application.metadataServicePort=20885 dubbo.registry.address=xds://istiod.istio-system.svc:15012 dubbo.consumer.timeout=3000 dubbo.consumer.check=false dubbo.application.qosEnable=true dubbo.application.qosAcceptForeignIp=true


4. Build docker images and deploy to kubernetes using proxyless mode.Here is the sample code: https://github.com/zewade/dubbo-samples-xds.git

### Expected Behavior

startupProbe/readinessProbe/livenessProbe success.

### Actual Behavior

startupProbe/readinessProbe/livenessProbe failed, because qosAcceptForeignIp is set to false in startQosServer finally in some case.

private void startQosServer(URL url) {
    try {
        if (!hasStarted.compareAndSet(false, true)) {
            return;
        }

        boolean qosEnable = url.getParameter(QOS_ENABLE, true);
        WireProtocol qosWireProtocol = frameworkModel.getExtensionLoader(WireProtocol.class).getExtension("qos");
        if(qosWireProtocol != null) {
            ((QosWireProtocol) qosWireProtocol).setQosEnable(qosEnable);
        }
        if (!qosEnable) {
            logger.info("qos won't be started because it is disabled. " +
                "Please check dubbo.application.qos.enable is configured either in system property, " +
                "dubbo.properties or XML/spring-boot configuration.");
            return;
        }

        String host = url.getParameter(QOS_HOST);
        int port = url.getParameter(QOS_PORT, QosConstants.DEFAULT_PORT);
        boolean acceptForeignIp = Boolean.parseBoolean(url.getParameter(ACCEPT_FOREIGN_IP, "false"));
        Server server = frameworkModel.getBeanFactory().getBean(Server.class);
        server.setHost(host);
        server.setPort(port);
        server.setAcceptForeignIp(acceptForeignIp);
        server.start();

    } catch (Throwable throwable) {
        logger.warn("Fail to start qos server: ", throwable);
    }
}

K8s Probe logs:

Startup probe failed: Get "http://10.234.226.117:22222/startup": net/http: HTTP/1.x transport connection broken: malformed HTTP response ""


Use `curl` to test starup probe.

[root@k8s-master1 k8s]# kubectl get pod dubbo-samples-xds-consumer-5b7647ddf6-25n79 -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES dubbo-samples-xds-consumer-5b7647ddf6-25n79 0/1 Running 0 54s 10.234.226.118 k8s-node1

[root@k8s-master1 k8s]# kubectl exec -it dubbo-samples-xds-consumer-5b7647ddf6-25n79 bash kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.

root@dubbo-samples-xds-consumer-5b7647ddf6-25n79:/# curl http://10.234.226.118:22222/startup curl: (1) Received HTTP/0.9 when not allowed

root@dubbo-samples-xds-consumer-5b7647ddf6-25n79:/# curl --http0.9 http://10.234.226.118:22222/startup

Foreign Ip Not Permitted.

root@dubbo-samples-xds-consumer-5b7647ddf6-25n79:/# curl http://localhost:22222/startup true root@dubbo-samples-xds-consumer-5b7647ddf6-25n79:/#


service log:

Listening for transport dt_socket at address: 31000 [09/09/22 03:30:14:014 UTC] main INFO annotation.AnnotationConfigApplicationContext: Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@6d21714c: startup date [Fri Sep 09 03:30:14 UTC 2022]; root of context hierarchy [09/09/22 03:30:14:014 UTC] main INFO model.FrameworkModel: [DUBBO] Reset global default framework from null to Dubbo Framework[1], dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO model.FrameworkModel: [DUBBO] Dubbo Framework[1] is created, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO model.ApplicationModel: [DUBBO] Dubbo Application1.0 is created, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO model.ScopeModel: [DUBBO] Dubbo Module[1.0.0] is created, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO context.AbstractConfigManager: [DUBBO] Config settings: {dubbo.config.mode=STRICT, dubbo.config.ignore-duplicated-interface=false}, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO context.AbstractConfigManager: [DUBBO] Config settings: {dubbo.config.mode=STRICT, dubbo.config.ignore-duplicated-interface=false}, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO model.FrameworkModel: [DUBBO] Reset global default application from null to Dubbo Application1.1, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO model.ApplicationModel: [DUBBO] Dubbo Application1.1 is created, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO model.ScopeModel: [DUBBO] Dubbo Module[1.1.0] is created, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO context.AbstractConfigManager: [DUBBO] Config settings: {dubbo.config.mode=STRICT, dubbo.config.ignore-duplicated-interface=false}, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO context.AbstractConfigManager: [DUBBO] Config settings: {dubbo.config.mode=STRICT, dubbo.config.ignore-duplicated-interface=false}, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO context.DubboSpringInitializer: [DUBBO] Use default application: Dubbo Application1.1, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO model.ScopeModel: [DUBBO] Dubbo Module[1.1.1] is created, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO context.AbstractConfigManager: [DUBBO] Config settings: {dubbo.config.mode=STRICT, dubbo.config.ignore-duplicated-interface=false}, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO context.DubboSpringInitializer: [DUBBO] Use default module model of target application: Dubbo Module[1.1.1], dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO context.DubboSpringInitializer: [DUBBO] Bind Dubbo Module[1.1.1] to spring container: org.springframework.beans.factory.support.DefaultListableBeanFactory@20e2cbe0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO annotation.ServiceAnnotationPostProcessor: [DUBBO] BeanNameGenerator bean can't be found in BeanFactory with name [org.springframework.context.annotation.internalConfigurationBeanNameGenerator], dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO annotation.ServiceAnnotationPostProcessor: [DUBBO] BeanNameGenerator will be a instance of org.springframework.context.annotation.AnnotationBeanNameGenerator , it maybe a potential problem on bean name generation., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO annotation.ServiceAnnotationPostProcessor: [DUBBO] Found 1 classes annotated by Dubbo @Service under package [org.apache.dubbo.samples.action]: [org.apache.dubbo.samples.action.SimpleMathServiceImpl], dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO annotation.ServiceAnnotationPostProcessor: [DUBBO] Register ServiceBean[ServiceBean:org.apache.dubbo.samples.api.SimpleMathService:1.0.0:]: Root bean: class [org.apache.dubbo.config.spring.ServiceBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:14:014 UTC] main INFO annotation.ReferenceAnnotationBeanPostProcessor: Register dubbo reference bean: greetingService = ReferenceBean:org.apache.dubbo.samples.api.GreetingService:1.0.0(providedBy=[dubbo-samples-xds-provider]) at private org.apache.dubbo.samples.api.GreetingService org.apache.dubbo.samples.action.GreetingServiceConsumer.greetingService [09/09/22 03:30:15:015 UTC] main INFO annotation.ReferenceAnnotationBeanPostProcessor: class org.apache.dubbo.config.spring.beans.factory.annotation.ReferenceAnnotationBeanPostProcessor was destroying! [09/09/22 03:30:15:015 UTC] main INFO context.DubboConfigBeanInitializer: loading dubbo config beans ... [09/09/22 03:30:15:015 UTC] main INFO context.DubboConfigBeanInitializer: dubbo config beans are loaded. [09/09/22 03:30:15:015 UTC] main INFO deploy.DefaultApplicationDeployer: [DUBBO] No value is configured in the registry, the DynamicConfigurationFactory extension[name : xds] does not support as the config center, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:15:015 UTC] main INFO deploy.DefaultApplicationDeployer: [DUBBO] The registry[] will be not used as the config center, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:15:015 UTC] main INFO deploy.DefaultModuleDeployer: [DUBBO] Dubbo Module[1.1.0] has been initialized!, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:15:015 UTC] main INFO deploy.DefaultModuleDeployer: [DUBBO] Dubbo Module[1.1.1] has been initialized!, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:15:015 UTC] main INFO deploy.DefaultApplicationDeployer: [DUBBO] No value is configured in the registry, the MetadataReportFactory extension[name : xds] does not support as the metadata center, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:15:015 UTC] main INFO deploy.DefaultApplicationDeployer: [DUBBO] The registry[] will be not used as the metadata center, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:15:015 UTC] main INFO deploy.DefaultApplicationDeployer: [DUBBO] Dubbo Application1.1 has been initialized!, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:15:015 UTC] main INFO reference.ReferenceCreator: The configBean[type:ReferenceConfig] has been built. [09/09/22 03:30:15:015 UTC] main INFO deploy.DefaultModuleDeployer: [DUBBO] Dubbo Module[1.1.1] is starting., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:15:015 UTC] main INFO deploy.DefaultApplicationDeployer: [DUBBO] Dubbo Application1.1 is starting., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:15:015 UTC] main WARN config.Environment: [DUBBO] dynamicConfiguration is null , return globalConfiguration., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:16:016 UTC] main INFO server.Server: [DUBBO] qos-server bind localhost:22222, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:16:016 UTC] main INFO config.ServiceConfig: [DUBBO] Export dubbo service org.apache.dubbo.samples.api.SimpleMathService to local registry url : injvm://127.0.0.1/org.apache.dubbo.samples.api.SimpleMathService?anyhost=true&application=dubbo-samples-xds-consumer&background=false&bind.ip=10.234.226.117&bind.port=50051&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=org.apache.dubbo.samples.api.SimpleMathService&metadata-service-port=20885&methods=add&pid=8&qos.accept.foreign.ip=true&qos.enable=true&release=1.0-SNAPSHOT&revision=1.0-SNAPSHOT&side=provider&timestamp=1662694215661&version=1.0.0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:16:016 UTC] main INFO config.ServiceConfig: [DUBBO] Register dubbo service org.apache.dubbo.samples.api.SimpleMathService url tri://10.234.226.117:50051/org.apache.dubbo.samples.api.SimpleMathService?anyhost=true&application=dubbo-samples-xds-consumer&background=false&bind.ip=10.234.226.117&bind.port=50051&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=org.apache.dubbo.samples.api.SimpleMathService&metadata-service-port=20885&methods=add&pid=8&qos.accept.foreign.ip=true&qos.enable=true&release=1.0-SNAPSHOT&revision=1.0-SNAPSHOT&service-name-mapping=true&side=provider&timestamp=1662694215661&version=1.0.0 to registry istiod.istio-system.svc:15012, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:16:016 UTC] main INFO transport.AbstractServer: [DUBBO] Start NettyPortUnificationServer bind /0.0.0.0:50051, export /0:0:0:0:0:0:0:0:50051, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:16:016 UTC] main INFO store.MetaCacheManager: [DUBBO] Successfully loaded mapping cache from file .metadata.dubbo-samples-xds-consumer.xds.istiod.istio-system.svc:15012, entries 0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] grpc-default-executor-0 INFO protocol.AbstractProtocol: [DUBBO] receive notification from xds server, type: type.googleapis.com/envoy.config.listener.v3.Listener requestId: 0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] grpc-default-executor-0 INFO protocol.AbstractProtocol: [DUBBO] receive notification from xds server, type: type.googleapis.com/envoy.config.route.v3.RouteConfiguration requestId: 1, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] grpc-default-executor-0 INFO protocol.AbstractProtocol: [DUBBO] receive notification from xds server, type: type.googleapis.com/envoy.config.route.v3.RouteConfiguration requestId: 3, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] grpc-default-executor-0 INFO protocol.AbstractProtocol: [DUBBO] receive notification from xds server, type: type.googleapis.com/envoy.config.listener.v3.Listener requestId: 5, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO metadata.MappingCacheManager: [DUBBO] Successfully loaded mapping cache from file .mapping.dubbo-samples-xds-consumer, entries 0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main WARN migration.MigrationRuleListener: [DUBBO] Using default configuration rule because config center is not configured!, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO config.ServiceConfig: [DUBBO] Register dubbo service org.apache.dubbo.samples.api.SimpleMathService url tri://10.234.226.117:50051/org.apache.dubbo.samples.api.SimpleMathService?anyhost=true&application=dubbo-samples-xds-consumer&background=false&bind.ip=10.234.226.117&bind.port=50051&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=org.apache.dubbo.samples.api.SimpleMathService&metadata-service-port=20885&methods=add&pid=8&qos.accept.foreign.ip=true&qos.enable=true&release=1.0-SNAPSHOT&revision=1.0-SNAPSHOT&service-name-mapping=true&side=provider&timestamp=1662694215661&version=1.0.0 to registry istiod.istio-system.svc:15012, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO xds.XdsRegistry: [DUBBO] Register: tri://10.234.226.117:50051/org.apache.dubbo.samples.api.SimpleMathService?anyhost=true&application=dubbo-samples-xds-consumer&background=false&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=org.apache.dubbo.samples.api.SimpleMathService&metadata-service-port=20885&methods=add&pid=8&release=1.0-SNAPSHOT&revision=1.0-SNAPSHOT&service-name-mapping=true&side=provider&timestamp=1662694215661&version=1.0.0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO xds.XdsRegistry: [DUBBO] Subscribe: provider://10.234.226.117:50051/org.apache.dubbo.samples.api.SimpleMathService?anyhost=true&application=dubbo-samples-xds-consumer&background=false&bind.ip=10.234.226.117&bind.port=50051&category=configurators&check=false&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=org.apache.dubbo.samples.api.SimpleMathService&metadata-service-port=20885&methods=add&pid=8&qos.accept.foreign.ip=true&qos.enable=true&release=1.0-SNAPSHOT&revision=1.0-SNAPSHOT&service-name-mapping=true&side=provider&timestamp=1662694215661&version=1.0.0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main WARN metadata.MetadataUtils: [DUBBO] Remote Metadata Report Server is not provided or unavailable, will stop registering service definition to remote center!, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main WARN metadata.MetadataServiceNameMapping: [DUBBO] No valid metadata config center found for mapping report., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main ERROR config.ServiceConfig: [DUBBO] Failed register interface application mapping for service org.apache.dubbo.samples.api.SimpleMathService:1.0.0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO deploy.DefaultModuleDeployer: [DUBBO] Dubbo Module[1.1.0] is starting., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO deploy.DefaultModuleDeployer: [DUBBO] Dubbo Module[1.1.0] has started., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO metadata.MetadataServiceNameMapping: [DUBBO] org.apache.dubbo.samples.api.GreetingService mapping to dubbo-samples-xds-provider instructed by provided-by set by user., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main WARN config.Environment: [DUBBO] dynamicConfiguration is null , return globalConfiguration., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main WARN config.Environment: [DUBBO] dynamicConfiguration is null , return globalConfiguration., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO xds.XdsRegistry: [DUBBO] Subscribe: consumer://10.234.226.117/org.apache.dubbo.samples.api.GreetingService?application=dubbo-samples-xds-consumer&background=false&category=providers,configurators,routers&check=false&dubbo=2.0.2&interface=org.apache.dubbo.samples.api.GreetingService&metadata-service-port=20885&methods=sayHello&pid=8&provided-by=dubbo-samples-xds-provider&qos.accept.foreign.ip=true&qos.enable=true&release=1.0-SNAPSHOT&revision=1.0-SNAPSHOT&side=consumer&sticky=false&timeout=3000&timestamp=1662694217536&version=1.0.0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO client.ServiceDiscoveryRegistry: [DUBBO] Trying to subscribe from apps dubbo-samples-xds-provider for service key org.apache.dubbo.samples.api.GreetingService:1.0.0, , dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] grpc-default-executor-0 INFO protocol.AbstractProtocol: [DUBBO] receive notification from xds server, type: type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment requestId: 6, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO model.ModuleModel: [DUBBO] Dynamically registering consumer model dubbo-samples-xds-provider/org.apache.dubbo.metadata.MetadataService:1.0.0 into model Dubbo Module[1.1.0], dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] NettyClientWorker-6-1 INFO netty4.NettyClientHandler: [DUBBO] The connection of /10.234.226.117:41286 -> /10.234.24.54:20885 is established., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO transport.AbstractClient: [DUBBO] Successfully connect to server /10.234.24.54:20885 from NettyClient 10.234.226.117 using dubbo version 1.0-SNAPSHOT, channel is NettyChannel [channel=[id: 0xf201d757, L:/10.234.226.117:41286 - R:/10.234.24.54:20885]], dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO transport.AbstractClient: [DUBBO] Start NettyClient /10.234.226.117 connect to the server /10.234.24.54:20885, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] NettyClientWorker-6-1 ERROR dubbo.CallbackServiceCodec: [DUBBO] Unable to get Service Model from Invocation. Please check if your invocation failed! This error only happen in UT cases! Invocation:RpcInvocation [methodName=getAndListenInstanceMetadata, parameterTypes=[class java.lang.String, interface org.apache.dubbo.metadata.InstanceMetadataChangedListener]], dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] NettyClientWorker-6-1 INFO dubbo.CallbackServiceCodec: [DUBBO] Export a callback service :dubbo://10.234.226.117:41286/org.apache.dubbo.metadata.InstanceMetadataChangedListener.25419362?getAndListenInstanceMetadata.1.callback=true&group=dubbo-samples-xds-provider&interface=org.apache.dubbo.metadata.InstanceMetadataChangedListener&is_callback_service=true&isserver=false&methods=echo,onEvent&reconnect=false&retries=0&side=consumer&timeout=5000, on NettyChannel [channel=[id: 0xf201d757, L:/10.234.226.117:41286 - R:/10.234.24.54:20885]], url is: dubbo://10.234.24.54:20885/org.apache.dubbo.metadata.MetadataService?getAndListenInstanceMetadata.1.callback=true&group=dubbo-samples-xds-provider&reconnect=false&retries=0&side=consumer&timeout=5000&version=1.0.0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO listener.ServiceInstancesChangedListener: [DUBBO] Received instance notification, serviceName: dubbo-samples-xds-provider, instances: 1, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO model.ModuleModel: [DUBBO] Dynamically registering consumer model dubbo-samples-xds-provider/org.apache.dubbo.metadata.MetadataService:1.0.0 into model Dubbo Module[1.1.0], dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO transport.AbstractClient: [DUBBO] Successfully connect to server /10.234.24.54:20885 from NettyClient 10.234.226.117 using dubbo version 1.0-SNAPSHOT, channel is NettyChannel [channel=[id: 0xd814c283, L:/10.234.226.117:41290 - R:/10.234.24.54:20885]], dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] NettyClientWorker-6-2 INFO netty4.NettyClientHandler: [DUBBO] The connection of /10.234.226.117:41290 -> /10.234.24.54:20885 is established., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO transport.AbstractClient: [DUBBO] Start NettyClient /10.234.226.117 connect to the server /10.234.24.54:20885, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO netty4.NettyChannel: [DUBBO] Close netty channel [id: 0xd814c283, L:/10.234.226.117:41290 - R:/10.234.24.54:20885], dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] NettyClientWorker-6-2 INFO netty4.NettyClientHandler: [DUBBO] The connection of /10.234.226.117:41290 -> /10.234.24.54:20885 is disconnected., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO listener.ServiceInstancesChangedListener: [DUBBO] 1 unique working revisions: f77ef326f9dbe8b0c8f75ee4ff53de93 , dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO listener.ServiceInstancesChangedListener: [DUBBO] Notify serviceKey: org.apache.dubbo.samples.api.GreetingService:1.0.0:null, listener: org.apache.dubbo.registry.client.ServiceDiscoveryRegistryDirectory@367b22e5 with 1 urls on subscription, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO client.ServiceDiscoveryRegistryDirectory: [DUBBO] Refreshed invoker size 1, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] grpc-default-executor-0 INFO protocol.AbstractProtocol: [DUBBO] receive notification from xds server, type: type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment requestId: 8, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO listener.ServiceInstancesChangedListener: [DUBBO] Received instance notification, serviceName: dubbo-samples-xds-provider, instances: 1, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO listener.ServiceInstancesChangedListener: [DUBBO] 1 unique working revisions: f77ef326f9dbe8b0c8f75ee4ff53de93 , dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO listener.ServiceInstancesChangedListener: [DUBBO] Notify service org.apache.dubbo.samples.api.GreetingService:1.0.0:null with urls 1, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO client.ServiceDiscoveryRegistryDirectory: [DUBBO] Refreshed invoker size 1, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO migration.DefaultMigrationAddressComparator: [DUBBO] No interface address available, stop compare., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO migration.MigrationRuleHandler: [DUBBO] Succeed Migrated to APPLICATION_FIRST mode. Service Name: org.apache.dubbo.samples.api.GreetingService:1.0.0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO config.ReferenceConfig: [DUBBO] Referred dubbo service: [org.apache.dubbo.samples.api.GreetingService]. it's not GenericService reference, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main WARN metadata.MetadataUtils: [DUBBO] Remote Metadata Report Server is not provided or unavailable, will stop registering service definition to remote center!, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO deploy.DefaultModuleDeployer: [DUBBO] Dubbo Module[1.1.1] has started., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO metadata.ConfigurableMetadataServiceExporter: [DUBBO] Using dubbo protocol to export metadata service on port 20885, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main WARN config.Environment: [DUBBO] dynamicConfiguration is null , return globalConfiguration., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:17:017 UTC] main INFO config.ServiceConfig: [DUBBO] Export dubbo service org.apache.dubbo.metadata.MetadataService to local registry url : injvm://127.0.0.1/org.apache.dubbo.metadata.MetadataService?anyhost=true&application=dubbo-samples-xds-consumer&background=false&bind.ip=10.234.226.117&bind.port=20885&connections=1&corethreads=2&delay=0&deprecated=false&dubbo=2.0.2&dynamic=true&executes=100&generic=false&getAndListenInstanceMetadata.1.callback=true&getAndListenInstanceMetadata.return=true&getAndListenInstanceMetadata.sent=true&group=dubbo-samples-xds-consumer&interface=org.apache.dubbo.metadata.MetadataService&metadata-service-port=20885&methods=getMetadataURL,isMetadataService,getExportedURLs,getAndListenInstanceMetadata,serviceName,version,getSubscribedURLs,getExportedServiceURLs,exportInstanceMetadata,getMetadataInfo,toSortedStrings,getMetadataInfos,getServiceDefinition,getInstanceMetadataChangedListenerMap&pid=8&qos.accept.foreign.ip=true&qos.enable=true&register=false&release=1.0-SNAPSHOT&revision=1.0-SNAPSHOT&side=provider&threadpool=cached&threads=100&timestamp=1662694217975&version=1.0.0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:18:018 UTC] main INFO config.ServiceConfig: [DUBBO] Export dubbo service org.apache.dubbo.metadata.MetadataService to url dubbo://10.234.226.117:20885/org.apache.dubbo.metadata.MetadataService?anyhost=true&application=dubbo-samples-xds-consumer&background=false&bind.ip=10.234.226.117&bind.port=20885&connections=1&corethreads=2&delay=0&deprecated=false&dubbo=2.0.2&dynamic=true&executes=100&generic=false&getAndListenInstanceMetadata.1.callback=true&getAndListenInstanceMetadata.return=true&getAndListenInstanceMetadata.sent=true&group=dubbo-samples-xds-consumer&interface=org.apache.dubbo.metadata.MetadataService&metadata-service-port=20885&methods=getMetadataURL,isMetadataService,getExportedURLs,getAndListenInstanceMetadata,serviceName,version,getSubscribedURLs,getExportedServiceURLs,exportInstanceMetadata,getMetadataInfo,toSortedStrings,getMetadataInfos,getServiceDefinition,getInstanceMetadataChangedListenerMap&pid=8&qos.accept.foreign.ip=true&qos.enable=true&register=false&release=1.0-SNAPSHOT&revision=1.0-SNAPSHOT&service-name-mapping=true&side=provider&threadpool=cached&threads=100&timestamp=1662694217975&version=1.0.0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:18:018 UTC] main INFO transport.AbstractServer: [DUBBO] Start NettyServer bind /0.0.0.0:20885, export /10.234.226.117:20885, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:18:018 UTC] main WARN migration.MigrationRuleListener: [DUBBO] Using default configuration rule because config center is not configured!, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:18:018 UTC] main INFO config.ServiceConfig: [DUBBO] Export dubbo service org.apache.dubbo.metadata.MetadataService to url dubbo://10.234.226.117:20885/org.apache.dubbo.metadata.MetadataService?anyhost=true&application=dubbo-samples-xds-consumer&background=false&bind.ip=10.234.226.117&bind.port=20885&connections=1&corethreads=2&delay=0&deprecated=false&dubbo=2.0.2&dynamic=true&executes=100&generic=false&getAndListenInstanceMetadata.1.callback=true&getAndListenInstanceMetadata.return=true&getAndListenInstanceMetadata.sent=true&group=dubbo-samples-xds-consumer&interface=org.apache.dubbo.metadata.MetadataService&metadata-service-port=20885&methods=getMetadataURL,isMetadataService,getExportedURLs,getAndListenInstanceMetadata,serviceName,version,getSubscribedURLs,getExportedServiceURLs,exportInstanceMetadata,getMetadataInfo,toSortedStrings,getMetadataInfos,getServiceDefinition,getInstanceMetadataChangedListenerMap&pid=8&qos.accept.foreign.ip=true&qos.enable=true&register=false&release=1.0-SNAPSHOT&revision=1.0-SNAPSHOT&service-name-mapping=true&side=provider&threadpool=cached&threads=100&timestamp=1662694217975&version=1.0.0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:18:018 UTC] main INFO xds.XdsRegistry: [DUBBO] Subscribe: provider://10.234.226.117:20885/org.apache.dubbo.metadata.MetadataService?anyhost=true&application=dubbo-samples-xds-consumer&background=false&bind.ip=10.234.226.117&bind.port=20885&category=configurators&check=false&connections=1&corethreads=2&delay=0&deprecated=false&dubbo=2.0.2&dynamic=true&executes=100&generic=false&getAndListenInstanceMetadata.1.callback=true&getAndListenInstanceMetadata.return=true&getAndListenInstanceMetadata.sent=true&group=dubbo-samples-xds-consumer&interface=org.apache.dubbo.metadata.MetadataService&metadata-service-port=20885&methods=getMetadataURL,isMetadataService,getExportedURLs,getAndListenInstanceMetadata,serviceName,version,getSubscribedURLs,getExportedServiceURLs,exportInstanceMetadata,getMetadataInfo,toSortedStrings,getMetadataInfos,getServiceDefinition,getInstanceMetadataChangedListenerMap&pid=8&qos.accept.foreign.ip=true&qos.enable=true&register=false&release=1.0-SNAPSHOT&revision=1.0-SNAPSHOT&service-name-mapping=true&side=provider&threadpool=cached&threads=100&timestamp=1662694217975&version=1.0.0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:18:018 UTC] main WARN metadata.MetadataServiceNameMapping: [DUBBO] No valid metadata config center found for mapping report., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:18:018 UTC] main ERROR config.ServiceConfig: [DUBBO] Failed register interface application mapping for service dubbo-samples-xds-consumer/org.apache.dubbo.metadata.MetadataService:1.0.0, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:18:018 UTC] main INFO metadata.ConfigurableMetadataServiceExporter: [DUBBO] The MetadataService exports urls : [dubbo://10.234.226.117:20885/org.apache.dubbo.metadata.MetadataService?anyhost=true&application=dubbo-samples-xds-consumer&background=false&bind.ip=10.234.226.117&bind.port=20885&connections=1&corethreads=2&delay=0&deprecated=false&dubbo=2.0.2&dynamic=true&executes=100&generic=false&getAndListenInstanceMetadata.1.callback=true&getAndListenInstanceMetadata.return=true&getAndListenInstanceMetadata.sent=true&group=dubbo-samples-xds-consumer&interface=org.apache.dubbo.metadata.MetadataService&metadata-service-port=20885&methods=getMetadataURL,isMetadataService,getExportedURLs,getAndListenInstanceMetadata,serviceName,version,getSubscribedURLs,getExportedServiceURLs,exportInstanceMetadata,getMetadataInfo,toSortedStrings,getMetadataInfos,getServiceDefinition,getInstanceMetadataChangedListenerMap&pid=8&qos.accept.foreign.ip=true&qos.enable=true&register=false&release=1.0-SNAPSHOT&revision=1.0-SNAPSHOT&service-name-mapping=true&side=provider&threadpool=cached&threads=100&timestamp=1662694217975&version=1.0.0], dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:18:018 UTC] main INFO metadata.ServiceInstanceMetadataUtils: [DUBBO] Start registering instance address to registry., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:18:018 UTC] main WARN metadata.ServiceInstanceHostPortCustomizer: [DUBBO] Can't find an instance URL using the default preferredProtocol "dubbo", falling back to the strategy that pick the first found protocol. Please try modifying the config of dubbo.application.protocol, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117, error code: 4-2. This may be caused by typo in preferred protocol, go to https://dubbo.apache.org/faq/4/2 to find instructions. [09/09/22 03:30:18:018 UTC] main INFO metadata.MetadataInfo: [DUBBO] metadata revision changed: null -> 97bf204ba6ca02e78019de79bb2ee0b5, app: dubbo-samples-xds-consumer, services: 1, dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 [09/09/22 03:30:18:018 UTC] main INFO deploy.DefaultApplicationDeployer: [DUBBO] Dubbo Application1.1 is ready., dubbo version: 1.0-SNAPSHOT, current host: 10.234.226.117 result: hello, xDS Consumer! from host: 10.234.24.54 result: hello, xDS Consumer! from host: 10.234.24.54 result: hello, xDS Consumer! from host: 10.234.24.54 result: hello, xDS Consumer! from host: 10.234.24.54 result: hello, xDS Consumer! from host: 10.234.24.54 result: hello, xDS Consumer! from host: 10.234.24.54 result: hello, xDS Consumer! from host: 10.234.24.54 result: hello, xDS Consumer! from host: 10.234.24.54 result: hello, xDS Consumer! from host: 10.234.24.54

AlbumenJ commented 2 years ago

Would you please help fix this?

yuchang01 commented 2 years ago

I has same issue, because my provider use tri protocol, next refer MetadataService use dubbo protocol, but url no qos.accept.foreign.ip=true param, the server changed server.setAcceptForeignIp(false);