【istio-1.16.5】serviceentry from meshregistry does not work with Istio virtualservice + destinationrule

tanjunchen commented 1 year ago

问题描述

[X] question 1：There are some problem with the host in serviceentry even though consumer-demo can access provider-demo through serviceentry. But the dr+vs strategy of the istio is not ok for consumer-demo. The dr+vs refer to https://github.com/tanjunchen/demo-registry2istio/tree/main/k8s. Related Issue: https://github.com/istio/istio/issues/45401. The routing of vs+dr will be not ok for the serviceentry service(No FQDN) from meshregistry. https://github.com/tanjunchen/demo-registry2istio/blob/main/k8s/demo-dr-vs.yaml

[ ] question 2：the logs of meshregistry alway output.(Not Good).

2023-07-21T07:17:22.573225Z info    mcp-xds Pushed networking.istio.io/v1beta1/ProxyConfig to sidecar~10.0.0.78~istiod-558df7d7cd-jz8nl.istio-system~istio-system.svc.cluster.local-1 count=0 size=0 nonce=
2023-07-21T07:17:22.573229Z info    mcp-xds recv req security.istio.io/v1beta1/AuthorizationPolicy from sidecar~10.0.0.78~istiod-558df7d7cd-jz8nl.istio-system~istio-system.svc.cluster.local-1, err <nil>, nonce
2023-07-21T07:17:22.573243Z info    mcp-xds Pushed security.istio.io/v1beta1/AuthorizationPolicy to sidecar~10.0.0.78~istiod-558df7d7cd-jz8nl.istio-system~istio-system.svc.cluster.local-1 count=0 size=0 nonce=
2023-07-21T07:17:22.573258Z info    mcp-xds recv req security.istio.io/v1beta1/PeerAuthentication from sidecar~10.0.0.78~istiod-558df7d7cd-jz8nl.istio-system~istio-system.svc.cluster.local-1, err <nil>, nonce
2023-07-21T07:17:22.573275Z info    mcp-xds Pushed security.istio.io/v1beta1/PeerAuthentication to sidecar~10.0.0.78~istiod-558df7d7cd-jz8nl.istio-system~istio-system.svc.cluster.local-1 count=0 size=0 nonce=
2023-07-21T07:17:22.573481Z info    mcp-xds recv req security.istio.io/v1beta1/RequestAuthentication from sidecar~10.0.0.78~istiod-558df7d7cd-jz8nl.istio-system~istio-system.svc.cluster.local-1, err <nil>, nonce
2023-07-21T07:17:22.573559Z info    mcp-xds Pushed security.istio.io/v1beta1/RequestAuthentication to sidecar~10.0.0.78~istiod-558df7d7cd-jz8nl.istio-system~istio-system.svc.cluster.local-1 count=0 size=0 nonce=
2023-07-21T07:17:22.573566Z info    mcp-xds recv req telemetry.istio.io/v1alpha1/Telemetry from sidecar~10.0.0.78~istiod-558df7d7cd-jz8nl.istio-system~istio-system.svc.cluster.local-1, err <nil>, nonce
2023-07-21T07:17:22.573595Z info    mcp-xds Pushed telemetry.istio.io/v1alpha1/Telemetry to sidecar~10.0.0.78~istiod-558df7d7cd-jz8nl.istio-system~istio-system.svc.cluster.local-1 count=0 size=0 nonce=
2023-07-21T07:17:22.573600Z info    mcp-xds recv req core/v1alpha1/MeshConfig from sidecar~10.0.0.78~istiod-558df7d7cd-jz8nl.istio-system~istio-system.svc.cluster.local-1, err <nil>, nonce
2023-07-21T07:17:22.573644Z info    mcp-xds Pushed core/v1alpha1/MeshConfig to sidecar~10.0.0.78~istiod-558df7d7cd-jz8nl.istio-system~istio-system.svc.cluster.local-1 count=0 size=0 nonce=
2023-07-21T07:17:22.573650Z info    mcp-xds recv req extensions.istio.io/v1alpha1/WasmPlugin from sidecar~10.0.0.78~istiod-558df7d7cd-jz8nl.istio-system~istio-system.svc.cluster.local-1, err <nil>, nonce

影响的子模块(在下面列表中打'X')

[ ] 配置懒加载
[ ] 插件管理
[ ] 智能限流
[X] 注册仓库
[ ] 安装Boot

重现问题的步骤

the source refer to https://github.com/tanjunchen/demo-registry2istio/tree/main/k8s

install meshregistry component according to https://slime-io.github.io/user-guide/meshregistry/tutorial/ I use the new image from this pr https://github.com/slime-io/slime/pull/405

apiVersion: config.netease.com/v1alpha1
kind: SlimeBoot
metadata:
name: meshregistry
namespace: mesh-operator
spec:
image:
pullPolicy: Always
repository: registry.baidubce.com/csm/slime-meshregistry
tag: fix-bug-841427a_linux_amd64-dirty_bcdc6b5
#repository: docker.io/slimeio/slime-meshregistry
#tag: v0.8.0
module:
- name: meshregistry
  kind: meshregistry
  enable: true
  general:
    LEGACY:
      NacosSource:
        Enabled: true
        RefreshPeriod: 30s
        Address:
          - "http://nacos:8848"
        Mode: polling

install istio (1.16.5) , the istio iop yaml:

https://github.com/tanjunchen/demo-registry2istio/blob/main/k8s/istio-config.yaml

install nacos

https://github.com/tanjunchen/demo-registry2istio/blob/main/k8s/nacos.yaml

deploy consumer-demo and provider-demo, the yaml：

https://github.com/tanjunchen/demo-registry2istio/blob/main/k8s/demo.yaml

the xdsCache from meshregistry: http://localhost:8081/meshregistry/xdsCache

{
"networking.istio.io/v1alpha3/ServiceEntry": [
{
  "type": "networking.istio.io/v1alpha3/ServiceEntry",
  "name": "consumer-demo",
  "namespace": "nacos",
  "labels": {
    "app": "consumer-demo",
    "registry": "nacos"
  },
  "annotations": {
    "ResourceVersion": "2023-07-21 06:18:25.025222868 +0000 UTC m=+35.665133534"
  },
  "creationTimestamp": "2023-07-21T06:18:25.025197428Z",
  "Spec": {
    "hosts": [
      "consumer-demo"
    ],
    "addresses": [],
    "ports": [
      {
        "number": 80,
        "protocol": "HTTP",
        "name": "http-80"
      },
      {
        "number": 9999,
        "protocol": "HTTP",
        "name": "http-9999"
      }
    ],
    "resolution": "STATIC",
    "endpoints": [
      {
        "address": "10.0.1.249",
        "ports": {
          "http-80": 9999,
          "http-9999": 9999
        },
        "labels": {
          "app": "consumer-demo",
          "istio-locality": "gz.zoneC",
          "pod-template-hash": "6478988b9b",
          "preserved.register.source": "SPRING_CLOUD",
          "security.istio.io/tlsMode": "istio",
          "service.istio.io/canonical-name": "consumer-demo",
          "service.istio.io/canonical-revision": "latest"
        },
        "locality": "gz/zoneC"
      }
    ]
  }
},
{
  "type": "networking.istio.io/v1alpha3/ServiceEntry",
  "name": "provider-demo",
  "namespace": "nacos",
  "labels": {
    "app": "provider-demo",
    "registry": "nacos"
  },
  "annotations": {
    "ResourceVersion": "2023-07-21 06:18:25.025246355 +0000 UTC m=+35.665157019"
  },
  "creationTimestamp": "2023-07-21T06:18:25.025236384Z",
  "Spec": {
    "hosts": [
      "provider-demo"
    ],
    "addresses": [],
    "ports": [
      {
        "number": 80,
        "protocol": "HTTP",
        "name": "http-80"
      },
      {
        "number": 10001,
        "protocol": "HTTP",
        "name": "http-10001"
      }
    ],
    "resolution": "STATIC",
    "endpoints": [
      {
        "address": "10.0.0.77",
        "ports": {
          "http-10001": 10001,
          "http-80": 10001
        },
        "labels": {
          "app": "provider-demo",
          "istio-locality": "gz.zoneC",
          "pod-template-hash": "768db54778",
          "preserved.register.source": "SPRING_CLOUD",
          "security.istio.io/tlsMode": "istio",
          "service.istio.io/canonical-name": "provider-demo",
          "service.istio.io/canonical-revision": "v2",
          "version": "v2"
        },
        "locality": "gz/zoneC"
      },
      {
        "address": "10.0.1.250",
        "ports": {
          "http-10001": 10001,
          "http-80": 10001
        },
        "labels": {
          "app": "provider-demo",
          "istio-locality": "gz.zoneC",
          "pod-template-hash": "7dd55b7994",
          "preserved.register.source": "SPRING_CLOUD",
          "security.istio.io/tlsMode": "istio",
          "service.istio.io/canonical-name": "provider-demo",
          "service.istio.io/canonical-revision": "v1",
          "version": "v1"
        },
        "locality": "gz/zoneC"
      }
    ]
  }
}
]
}

the configz of istiod, http://localhost:8080/debug/configz

{
"kind": "ServiceEntry",
"apiVersion": "networking.istio.io/v1alpha3",
"metadata": {
    "name": "provider-demo",
    "namespace": "nacos",
    "resourceVersion": "2023-07-21 06:20:15.277115637 +0000 UTC m=+9186.965321822",
    "creationTimestamp": "2023-07-21T06:18:25Z",
    "labels": {
        "app": "provider-demo",
        "registry": "nacos"
    },
    "annotations": {
        "ResourceVersion": "2023-07-21 06:18:25.025246355 +0000 UTC m=+35.665157019"
    }
},
"spec": {
    "endpoints": [{
            "address": "10.0.0.77",
            "labels": {
                "app": "provider-demo",
                "istio-locality": "gz.zoneC",
                "pod-template-hash": "768db54778",
                "preserved.register.source": "SPRING_CLOUD",
                "security.istio.io/tlsMode": "istio",
                "service.istio.io/canonical-name": "provider-demo",
                "service.istio.io/canonical-revision": "v2",
                "version": "v2"
            },
            "locality": "gz/zoneC",
            "ports": {
                "http-10001": 10001,
                "http-80": 10001
            }
        },
        {
            "address": "10.0.1.250",
            "labels": {
                "app": "provider-demo",
                "istio-locality": "gz.zoneC",
                "pod-template-hash": "7dd55b7994",
                "preserved.register.source": "SPRING_CLOUD",
                "security.istio.io/tlsMode": "istio",
                "service.istio.io/canonical-name": "provider-demo",
                "service.istio.io/canonical-revision": "v1",
                "version": "v1"
            },
            "locality": "gz/zoneC",
            "ports": {
                "http-10001": 10001,
                "http-80": 10001
            }
        }
    ],
    "hosts": [
        "provider-demo"
    ],
    "ports": [{
            "name": "http-80",
            "number": 80,
            "protocol": "HTTP"
        },
        {
            "name": "http-10001",
            "number": 10001,
            "protocol": "HTTP"
        }
    ],
    "resolution": "STATIC"
}
}

the config_dump of consumer-demo configdump.tar.gz

MouceL commented 1 year ago

if lazyload deployed in your local cluster

more info ..

consumer cnofigdump
attach accesslog in consumer's envoy

tanjunchen commented 1 year ago

if lazyload deployed in your local cluster

more info ..

consumer cnofigdump

attach accesslog in consumer's envoy

no lazyload
the config_dump of consumer configdump.tar.gz

the log


kubectl -n nacos get pod -owide
NAME                                READY   STATUS    RESTARTS   AGE     IP           NODE           NOMINATED NODE   READINESS GATES
consumer-demo-6478988b9b-bggks      2/2     Running   0          3h56m   10.0.1.249   192.168.1.17   <none>           <none>
provider-demo-v1-7dd55b7994-vd2ww   2/2     Running   0          3h56m   10.0.1.250   192.168.1.17   <none>           <none>
provider-demo-v2-768db54778-f6t5l   2/2     Running   0          3h56m   10.0.0.77    192.168.1.12   <none>           <none>
kubectl -n nacos  exec -it consumer-demo-6478988b9b-bggks  -c consumer-demo -- curl 10.0.1.249:9999/echo-rest/aaaaa
{"timestamp":"2023-07-21T07:38:44.387+0000","status":500,"error":"Internal Server Error","message":"I/O error on GET request for \"http://provider-demo/echo/aaaaa\": provider-demo; nested exception is java.net.UnknownHostException: provider-demo","path":"/echo-rest/aaaaa"}

2023-07-21T03:41:13.935087Z info 2023-07-21T03:41:13.935110Z info 2023-07-21T03:41:13.935116Z info 2023-07-21T03:41:13.935120Z info 2023-07-21T03:41:13.935123Z info 2023-07-21T03:41:13.935127Z info 2023-07-21T03:41:13.935130Z info 2023-07-21T03:41:13.935133Z info 2023-07-21T03:41:13.935137Z info 2023-07-21T03:41:13.935140Z info 2023-07-21T03:41:13.935144Z info 2023-07-21T03:41:13.935152Z info 2023-07-21T03:41:13.935156Z info 2023-07-21T03:41:13.935159Z info 2023-07-21T03:41:13.935162Z info 2023-07-21T03:41:13.935165Z info 2023-07-21T03:41:13.935168Z info 2023-07-21T03:41:13.935171Z info 2023-07-21T03:41:13.935174Z info 2023-07-21T03:41:13.935178Z info 2023-07-21T03:41:13.935186Z info 2023-07-21T03:41:13.935190Z info 2023-07-21T03:41:13.940898Z info 2023-07-21T03:41:13.941070Z info 2023-07-21T03:41:13.941153Z info FLAG: --concurrency="2" FLAG: --domain="nacos.svc.cluster.local" FLAG: --help="false" FLAG: --log_as_json="false" FLAG: --log_caller="" FLAG: --log_output_level="default:info" FLAG: --log_rotate="" FLAG: --log_rotate_max_age="30" FLAG: --log_rotate_max_backups="1000" FLAG: --log_rotate_max_size="104857600" FLAG: --log_stacktrace_level="default:none" FLAG: --log_target="[stdout]" FLAG: --meshConfig="./etc/istio/config/mesh" FLAG: --outlierLogPath="" FLAG: --proxyComponentLogLevel="misc:error" FLAG: --proxyLogLevel="warning" FLAG: --serviceCluster="istio-proxy" FLAG: --stsPort="0" FLAG: --templateFile="" FLAG: --tokenManagerPlugin="GoogleTokenExchange" FLAG: --vklog="0" Version 1.16.5-ae8d5164776cd55bf61d9d3fc4658b44a77c6e24-Clean Maximum file descriptors (ulimit -n): 1048576 Proxy role ips=[10.0.1.249] type=sidecar id=consumer-demo-6478988b9b-bggks.nacos domain=nacos.svc.cluster.local Apply proxy config from env {"proxyMetadata":{"ISTIO_META_DNS_CAPTURE":"true"},"holdApplicationUntilProxyStarts":true}

2023-07-21T03:41:13.958232Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod.istio-system.svc:15012 drainDuration: 45s holdApplicationUntilProxyStarts: true parentShutdownDuration: 60s proxyAdminPort: 15000 proxyMetadata: ISTIO_META_DNS_CAPTURE: "true" serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s tracing: zipkin: address: zipkin.istio-system:9411

2023-07-21T03:41:13.958250Z info JWT policy is third-party-jwt 2023-07-21T03:41:13.958255Z info using credential fetcher of JWT type in cluster.local trust domain 2023-07-21T03:41:13.969698Z info Opening status port 15020 2023-07-21T03:41:13.970524Z info dns Starting local udp DNS server on 127.0.0.1:15053 2023-07-21T03:41:13.970548Z info dns Starting local tcp DNS server on 127.0.0.1:15053 2023-07-21T03:41:13.970589Z info Workload SDS socket not found. Starting Istio SDS Server 2023-07-21T03:41:13.970601Z info CA Endpoint istiod.istio-system.svc:15012, provider Citadel 2023-07-21T03:41:13.970702Z info Using CA istiod.istio-system.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2023-07-21T03:41:13.970839Z info citadelclient Citadel client using custom root cert: var/run/secrets/istio/root-cert.pem 2023-07-21T03:41:14.083433Z info ads All caches have been synced up in 151.022806ms, marking server ready 2023-07-21T03:41:14.107073Z info xdsproxy Initializing with upstream address "istiod.istio-system.svc:15012" and cluster "Kubernetes" 2023-07-21T03:41:14.110256Z info sds Starting SDS grpc server 2023-07-21T03:41:14.113246Z info starting Http service at 127.0.0.1:15004 2023-07-21T03:41:14.132011Z info Pilot SAN: [istiod.istio-system.svc] 2023-07-21T03:41:14.160148Z info Starting proxy agent 2023-07-21T03:41:14.160283Z info starting 2023-07-21T03:41:14.160340Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --parent-shutdown-time-s 60 --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --log-format %Y-%m-%dT%T.%fZ %l envoy %n %v -l warning --component-log-level misc:error --concurrency 2] 2023-07-21T03:41:15.001595Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 2023-07-21T03:41:15.116036Z info ads ADS: new connection for node:consumer-demo-6478988b9b-bggks.nacos-1 2023-07-21T03:41:15.128098Z info ads ADS: new connection for node:consumer-demo-6478988b9b-bggks.nacos-2 2023-07-21T03:41:15.341959Z info cache generated new workload certificate latency=1.256913434s ttl=23h59m59.65805447s 2023-07-21T03:41:15.342044Z info cache Root cert has changed, start rotating root cert 2023-07-21T03:41:15.342081Z info ads XDS: Incremental Pushing:0 ConnectedEndpoints:2 Version: 2023-07-21T03:41:15.342242Z info cache returned workload trust anchor from cache ttl=23h59m59.657762469s 2023-07-21T03:41:15.342325Z info cache returned workload certificate from cache ttl=23h59m59.657677412s 2023-07-21T03:41:15.342744Z info ads SDS: PUSH request for node:consumer-demo-6478988b9b-bggks.nacos resources:1 size:4.0kB resource:default 2023-07-21T03:41:15.344105Z info cache returned workload trust anchor from cache ttl=23h59m59.655901036s 2023-07-21T03:41:15.344355Z info ads SDS: PUSH request for node:consumer-demo-6478988b9b-bggks.nacos resources:1 size:1.1kB resource:ROOTCA 2023-07-21T03:41:15.344474Z info cache returned workload trust anchor from cache ttl=23h59m59.65555317s 2023-07-21T03:41:15.593934Z info Readiness succeeded in 1.760746707s 2023-07-21T03:41:15.595222Z info Envoy proxy is ready [2023-07-21T03:41:31.888Z] - 0 - - "-" 1038 1285 72671 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:55570 172.16.242.146:8848 10.0.1.249:52796 - [2023-07-21T03:41:31.893Z] - 0 - - "-" 68274 42821 336196 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:55584 172.16.242.146:8848 10.0.1.249:52804 - 2023-07-21T03:47:10.674562Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 [2023-07-21T03:47:13.090Z] - 0 - - "-" 68125 42660 330370 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:39052 172.16.242.146:8848 10.0.1.249:53616 - [2023-07-21T03:52:45.119Z] - 0 - - "-" 68150 43193 330280 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:49068 172.16.242.146:8848 10.0.1.249:57158 - [2023-07-21T03:58:18.857Z] - 0 - - "-" 68125 42660 330380 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:42608 172.16.242.146:8848 10.0.1.249:41334 - [2023-07-21T04:03:54.239Z] - 0 - - "-" 68125 42660 330421 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:39908 172.16.242.146:8848 10.0.1.249:42628 - [2023-07-21T04:09:25.802Z] - 0 - - "-" 68150 43193 330193 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:38754 172.16.242.146:8848 10.0.1.249:52846 - 2023-07-21T04:17:34.774734Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 [2023-07-21T04:15:00.061Z] - 0 - - "-" 68125 42660 330343 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:44742 172.16.242.146:8848 10.0.1.249:54644 - [2023-07-21T04:20:35.406Z] - 0 - - "-" 68125 42660 330378 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:50650 172.16.242.146:8848 10.0.1.249:60368 - [2023-07-21T04:26:06.377Z] - 0 - - "-" 68150 43193 330232 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:34892 172.16.242.146:8848 10.0.1.249:51288 - [2023-07-21T04:31:41.103Z] - 0 - - "-" 68125 42660 330289 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:45710 172.16.242.146:8848 10.0.1.249:32904 - [2023-07-21T04:37:16.393Z] - 0 - - "-" 68125 42660 330296 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:39834 172.16.242.146:8848 10.0.1.249:52700 - [2023-07-21T04:42:47.007Z] - 0 - - "-" 68150 43193 330204 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:36108 172.16.242.146:8848 10.0.1.249:39940 - 2023-07-21T04:49:10.209851Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 [2023-07-21T04:48:22.051Z] - 0 - - "-" 68125 42660 330298 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:40500 172.16.242.146:8848 10.0.1.249:45424 - [2023-07-21T04:56:07.487Z] - 0 - - "-" 698 784 64919 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:47814 172.16.242.146:8848 10.0.1.249:41166 - [2023-07-21T04:53:57.348Z] - 0 - - "-" 68125 42660 330367 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:49098 172.16.242.146:8848 10.0.1.249:52802 - [2023-07-21T04:59:32.717Z] - 0 - - "-" 68125 42660 330391 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:32840 172.16.242.146:8848 10.0.1.249:37190 - [2023-07-21T05:05:07.794Z] - 0 - - "-" 68150 43193 330132 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:55224 172.16.242.146:8848 10.0.1.249:49070 - [2023-07-21T05:10:38.562Z] - 0 - - "-" 68125 42660 330333 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:44546 172.16.242.146:8848 10.0.1.249:39300 - 2023-07-21T05:17:16.210863Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 [2023-07-21T05:16:13.897Z] - 0 - - "-" 68125 42660 330401 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:53218 172.16.242.146:8848 10.0.1.249:54250 - [2023-07-21T05:21:48.305Z] - 0 - - "-" 68150 43193 330246 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:52968 172.16.242.146:8848 10.0.1.249:57040 - [2023-07-21T05:27:19.598Z] - 0 - - "-" 68125 42660 330378 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:46018 172.16.242.146:8848 10.0.1.249:47680 - [2023-07-21T05:32:54.978Z] - 0 - - "-" 68125 42660 330273 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:46944 172.16.242.146:8848 10.0.1.249:46104 - [2023-07-21T05:38:28.909Z] - 0 - - "-" 68150 43193 330195 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:37048 172.16.242.146:8848 10.0.1.249:53780 - 2023-07-21T05:48:56.383019Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 [2023-07-21T05:44:00.607Z] - 0 - - "-" 68125 42660 330340 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:52214 172.16.242.146:8848 10.0.1.249:40370 - [2023-07-21T05:49:35.953Z] - 0 - - "-" 68125 42660 330319 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:60994 172.16.242.146:8848 10.0.1.249:36712 - [2023-07-21T05:55:09.428Z] - 0 - - "-" 68150 43193 330230 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:55694 172.16.242.146:8848 10.0.1.249:41998 - [2023-07-21T06:00:41.663Z] - 0 - - "-" 68125 42660 330386 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:52376 172.16.242.146:8848 10.0.1.249:41498 - [2023-07-21T06:06:17.052Z] - 0 - - "-" 68125 42660 330138 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:59152 172.16.242.146:8848 10.0.1.249:56074 - 2023-07-21T06:16:59.914148Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 [2023-07-21T06:11:49.983Z] - 0 - - "-" 68150 43193 330055 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:37722 172.16.242.146:8848 10.0.1.249:33250 - [2023-07-21T06:17:22.280Z] - 0 - - "-" 68125 42660 330200 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:40196 172.16.242.146:8848 10.0.1.249:51002 - [2023-07-21T06:22:57.484Z] - 0 - - "-" 68125 42660 330329 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:34566 172.16.242.146:8848 10.0.1.249:44834 - [2023-07-21T06:28:30.382Z] - 0 - - "-" 68150 43193 330207 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:36998 172.16.242.146:8848 10.0.1.249:46546 - [2023-07-21T06:34:03.104Z] - 0 - - "-" 68125 42660 330295 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:51692 172.16.242.146:8848 10.0.1.249:33950 - [2023-07-21T06:39:38.401Z] - 0 - - "-" 68125 42660 330273 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:39130 172.16.242.146:8848 10.0.1.249:36850 - 2023-07-21T06:48:50.628103Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 [2023-07-21T06:45:10.905Z] - 0 - - "-" 68150 43193 330222 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:40244 172.16.242.146:8848 10.0.1.249:38390 - [2023-07-21T06:50:44.005Z] - 0 - - "-" 68125 42660 330314 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:56028 172.16.242.146:8848 10.0.1.249:52574 - [2023-07-21T06:56:19.320Z] - 0 - - "-" 68125 42660 330353 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:51254 172.16.242.146:8848 10.0.1.249:60854 - [2023-07-21T07:01:51.503Z] - 0 - - "-" 68150 43193 330217 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:46134 172.16.242.146:8848 10.0.1.249:41840 - [2023-07-21T07:07:25.002Z] - 0 - - "-" 68125 42660 330304 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:37214 172.16.242.146:8848 10.0.1.249:55542 - 2023-07-21T07:18:11.613611Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 [2023-07-21T07:13:00.307Z] - 0 - - "-" 68125 42660 330294 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:33836 172.16.242.146:8848 10.0.1.249:58782 - [2023-07-21T07:18:32.075Z] - 0 - - "-" 68150 43193 330173 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:44746 172.16.242.146:8848 10.0.1.249:49730 - [2023-07-21T07:24:05.904Z] - 0 - - "-" 68125 42660 330310 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:40390 172.16.242.146:8848 10.0.1.249:49238 - [2023-07-21T07:29:41.215Z] - 0 - - "-" 68125 42660 330332 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:59814 172.16.242.146:8848 10.0.1.249:50818 - [2023-07-21T07:35:12.606Z] - 0 - - "-" 68150 43193 330149 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:53762 172.16.242.146:8848 10.0.1.249:33950 - 2023-07-21T07:45:39.516213Z info xdsproxy connected to upstream XDS server: istiod.istio-system.svc:15012 [2023-07-21T07:40:46.892Z] - 0 - - "-" 68125 42660 330303 "-" "-" outbound|8848||nacos.mesh-operator.svc.cluster.local 10.0.1.249:54156 172.16.242.146:8848 10.0.1.249:57386 - ``

MouceL commented 1 year ago

kubectl -n nacos exec -it consumer-demo-6478988b9b-bggks -c consumer-demo

kubectl -n nacos exec -it consumer-demo-6478988b9b-bggks -c istio-proxy

tanjunchen commented 1 year ago

kubectl -n nacos exec -it consumer-demo-6478988b9b-bggks -c consumer-demo

kubectl -n nacos exec -it consumer-demo-6478988b9b-bggks -c istio-proxy

done.

MouceL commented 1 year ago

看configdump, provider 的所有配置都下发了

但是consumer的accesslog里怎么没有访问记录呢

你们只能dns开了吗

lds

62E3FA01-1E90-419F-8CA6-FB2FA9E7976C

cds

CF456AD8-48EB-4EB9-963A-A4B714B28D8E

rds 90358632-45F4-46B8-AAF1-9B3F34E0F8BF

tanjunchen commented 1 year ago

@MouceL question 1 have been resolved. It seems that we must use smart DNS. Smart DNS must add ISTIO_META_DNS_AUTO_ALLOCATE=true in proxyMetadata. ok：

proxyMetadata:
  ISTIO_META_DNS_AUTO_ALLOCATE: "true"
  ISTIO_META_DNS_CAPTURE: "true"

fail：

proxyMetadata:
  ISTIO_META_DNS_CAPTURE: "true"

apiVersion: v1
data:
  mesh: |-
    accessLogFile: /dev/stdout
    accessLogFormat: |
      [%START_TIME%] %REQ(X-META-PROTOCOL-APPLICATION-PROTOCOL)% %RESPONSE_CODE% %RESPONSE_CODE_DETAILS% %CONNECTION_TERMINATION_DETAILS% "%UPSTREAM_TRANSPORT_FAILURE_REASON%" %BYTES_RECEIVED% %BYTES_SENT% %DURATION% "%REQ(X-FORWARDED-FOR)%" "%REQ(X-REQUEST-ID)%" %UPSTREAM_CLUSTER% %UPSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_REMOTE_ADDRESS% %ROUTE_NAME%
    configSources:
    - address: k8s://
    - address: xds://meshregistry.mesh-operator.svc:16010
    defaultConfig:
      discoveryAddress: istiod.istio-system.svc:15012
      holdApplicationUntilProxyStarts: true
      proxyMetadata:
        ISTIO_META_DNS_AUTO_ALLOCATE: "true"
        ISTIO_META_DNS_CAPTURE: "true"
      tracing:
        zipkin:
          address: zipkin.istio-system:9411
    enablePrometheusMerge: true
    enableTracing: true
    rootNamespace: istio-system
    trustDomain: cluster.local
  meshNetworks: 'networks: {}'
kind: ConfigMap

tanjunchen commented 1 year ago

@MouceL Can you help confirm that vs+dr cannot take effect? I think this is a bug, ServiceEntry does the host need to fill in the FQDN.

MouceL commented 1 year ago

@MouceL Can you help confirm that vs+dr cannot take effect? I think this is a bug, ServiceEntry does the host need to fill in the FQDN.

the meshregistry is only responsible for transforming services into serviceentry.

it's istiod's feature When short names are used (e.g. “reviews” instead of “reviews.default.svc.cluster.local”), Istio will interpret the short name based on the namespace of the rule, not the service. A rule in the “default” namespace containing a host “reviews” will be interpreted as “reviews.default.svc.cluster.local”, irrespective of the actual namespace associated with the reviews service.

https://istio.io/latest/docs/reference/config/networking/virtual-service/#VirtualService

MouceL commented 1 year ago

Contributor

@MouceL Can you help confirm that vs+dr cannot take effect? I think this is a bug, ServiceEntry does the host need to fill in the FQDN.

you can add dots in domains to resolve it，refer to

https://github.com/istio/istio/blob/master/pilot/pkg/model/config.go#L226

slime-io / slime

【istio-1.16.5】serviceentry from meshregistry does not work with Istio virtualservice + destinationrule #408