Allow tidb service to have multiple service port

[wxiaomou]

Feature Request

Is your feature request related to a problem? Please describe:

We use service mesh to route traffic, and we need to have multiple service ports from tidb. For example, 1 port for az aware routing and 1 port for az unaware routing. In this case we need to add multiple service ports in the tidb service.

Describe the feature you'd like:

Allow tidb operator to set multiple service port in the tidb service.

Currently we can specify the service port in spec->tidb->service->port. But it just support 1 port. We need to support multiple service ports in our internal use case. I feel it's a reasonable feature to add,

Describe alternatives you've considered:

N/A Teachability, Documentation, Adoption, Migration Strategy:

[yiduoyunQ]

does heterogeneous-tidb-cluster meet your needs? It deploy separate tidb pods and service which is isolated from original ones.

[wxiaomou]

@yiduoyunQ we have deployed heterogeneous-tidb-cluster, we can use the cross-az port for that cluster, but it limited our flexibility on the cluster configuration. For example, we deployed tidb into 3 AZs. We have a main tidb cluster that has 90 pods in total that surviving online read/write from our customer, meanwhile we have 45 pod in the heterogeneous-tidb-cluster serving some internal write operation like enforcing TTL etc. And our internal write operation is expensive so that we want to use cross-az routing in this case to avoid overloading tidb in one AZ. We can use different port for these two clusters now, but it will limited the flexibility of how we using these tidb clusters in the future. Also given the cluster size, it's not ideal if we just add a new heterogeneous-tidb-cluster for a new use case. Does this make sense to you?

[yiduoyunQ]

main cluster and heterogeneous cluster share the same TiKV level and PD cluster, you can deploy only heterogeneous tidb component pod in heterogeneous cluster to scale cross-az or single az entrance point. meanwhile you need maintain 2 tidb cluster which will bring more maintain work.

[wxiaomou]

Yes. But we want to have both az and cross-az workload on the main tidb and heterogeneous tidb cluster. This is needed based on our business logic.

[yiduoyunQ]

for example, you can deploy like this:

main cluster: 3 PD, 90 TiKV, 2 TiDB (1 single AZ LB) heterogeneous cluster: no PD, 45 TiKV, 2 TiDB (1 cross AZ LB)

You need know that both main cluster and heterogeneous clusters are grouping to a big tidb cluster: 3 PD, 135 TiKV, 2 TiDB (1 single AZ LB entry), 2 TiDB (1 cross AZ LB entry), so both az and cross-az workload can be on the 90 TiKV in main cluster and 45 TiKV in heterogeneous tidb cluster.

The heterogeneous cluster is a little similar to scale out the original main cluster, but it is more flexible about adding one more entry for cluster, or some other Usage scenarios.

[wxiaomou]

Yes, but we want to have the 2 TiDB in the main cluster to have both single AZ LB and cross AZ LB. And we uses different port for them.

[csuzhangxc]

Cloud you add an additional service outside of TiDB Operator?

[wxiaomou]

@csuzhangxc yes, we have a discussion with @hanlins , agreed having an additional service is better solution, I'm closing this issue. Thanks