request help: How to route according to the configuration of nacos metadata?

[alex8224]

Issue description

I use nacos discovery, and for each microservice I register some additional metadata for finer-grained control

I know that the traffic-split plugin can match routes at a certain level of granularity As far as I know, this plugin can't match upstreams based on metadata level

So, I would like to ask how to do this

Tks

Environment

• apisix 2.10.2
• Centos7
• openresty/1.19.3.2
• etcd 3.5.0

[tzssangglass]

I know that the traffic-split plugin can match routes at a certain level of granularity

the traffic-split plugin is not involved in route matching.

Are you trying to select upstream based on the additional information obtained in nacos?

[tokers]

@alex8224 You may implement your own balancer and use it in Upstream objects.

[tokers]

@alex8224 You may implement your own balancer and use it in Upstream objects.

See https://apisix.apache.org/blog/2021/07/28/release-apache-apisix-2.8/#new-way-to-develop-support-for-custom-balancer .

[alex8224]

@tzssangglass yes, I want to select upstream based on metadata obtained in nacos.

[alex8224]

@alex8224 You may implement your own balancer and use it in Upstream objects.

See https://apisix.apache.org/blog/2021/07/28/release-apache-apisix-2.8/#new-way-to-develop-support-for-custom-balancer . @tokers I can only find very little about how to implement a balancer from the url you gave, are there other more detailed guides on how to easily add a balancer type? For example.

1. at what stage does the nacos metadata information get and where is it placed?
2. where to add require("apisix.balancer.your_balancer")?
3. what functions should I implement in my custom balancer?

[tzssangglass]

1. at what stage does the nacos metadata information get and where is it placed?

You can see how the service registration information for nacos is obtained and how it is stored in nacos.lua. e.g. https://github.com/apache/apisix/blob/master/apisix/discovery/nacos/init.lua#L216-L230

2. where to add require("apisix.balancer.your_balancer")?

your_balancer is placed in the apisix/balancer folder

3. what functions should I implement in my custom balancer?

take a look at: https://github.com/apache/apisix/blob/master/apisix/balancer/roundrobin.lua

BTW, I think this is a bit difficult, if you haven't had experience developing plugins before.

[xuminwlt]

+1 need too.

[tzssangglass]

Maybe we can enhance nacos discovery to make it work. Would anyone like to give a detailed example? For example, from which interface of nacos do we get information? What information is being fetched?

[alex8224]

@tzssangglass If so, that's pretty good

Generally use the 127.0.0.1:8848/nacos/v1/ns/instance/list?serviceName=xxx&namespaceId=dev interface to get all service instances under a namespaceId, The output of the interface call is as follows

{
   "allIPs":false,
   "cacheMillis":10000,
   "checksum":"",
   "clusters":"",
   "groupName":"DEFAULT_GROUP",
   "hosts":[
      {
         "clusterName":"DEFAULT",
         "enabled":true,
         "ephemeral":true,
         "healthy":true,
         "instanceHeartBeatInterval":5000,
         "instanceHeartBeatTimeOut":15000,
         "instanceId":"192.26.0.7#9088#DEFAULT#DEFAULT_GROUP@@xxx",
         "ip":"192.26.0.7",
         "ipDeleteTimeout":30000,
         "metadata":{
            "app.name":"xxx",
            "preserved.register.source":"SPRING_CLOUD",
            "startup.time":"1638849194096",
            "dept":"HR"
         },
         "port":9088,
         "serviceName":"DEFAULT_GROUP@@xxx",
         "weight":1.0
      },
      {
         "clusterName":"DEFAULT",
         "enabled":true,
         "ephemeral":true,
         "healthy":true,
         "instanceHeartBeatInterval":5000,
         "instanceHeartBeatTimeOut":15000,
         "instanceId":"192.26.0.6#9088#DEFAULT#DEFAULT_GROUP@@xxx",
         "ip":"192.26.0.6",
         "ipDeleteTimeout":30000,
         "metadata":{
            "app.name":"xxx",
            "preserved.register.source":"SPRING_CLOUD",
            "startup.time":"1638849194096",
            "dept":"IT"
         },
         "port":9088,
         "serviceName":"DEFAULT_GROUP@@xxx",
         "weight":1.0
      },
      {
         "clusterName":"DEFAULT",
         "enabled":true,
         "ephemeral":true,
         "healthy":true,
         "instanceHeartBeatInterval":5000,
         "instanceHeartBeatTimeOut":15000,
         "instanceId":"192.26.0.9#9088#DEFAULT#DEFAULT_GROUP@@xxx",
         "ip":"192.26.0.9",
         "ipDeleteTimeout":30000,
         "metadata":{
            "app.name":"xxx",
            "preserved.register.source":"SPRING_CLOUD",
            "startup.time":"1638849194096",
            "dept":"HRP"
         },
         "port":9088,
         "serviceName":"DEFAULT_GROUP@@xxx",
         "weight":1.0
      }
   ],
   "lastRefTime":1638890095796,
   "name":"DEFAULT_GROUP@@bcs-server",
   "reachProtectionThreshold":false,
   "valid":true
}```

There are several scenarios as follows
route url = /xxx/*
When the client requests /xxx/*
When header ["X-Real-Ip"] between ["192.168.1-192.168.1.10"], the request is routed to the instance node with metadata field dept == HR, in this case node 192.26.0.7
Other requests that do not match are routed to nodes that do not match, in this case to 192.26.0.6 and 192.26.0.9

[wingyiu]

+1 route to nodes by match meta label

[jzhao20230918]

It would be great if we can have this feature.

[yangxiangyu]

+1

[ddavidzhang]

+1