TiDB Cluster/Client TLS support implementation in TiDB Operator

[weekface]

To support various certificate issuers and methods, for example: user-defined certificates, K8s builtin CA system or cert-manager, we will refactor the TiDB Cluster/Client TLS feature with new API and new usage.

First, we will change the Cluster/Client TLS API to:

tlsCluster:
  enabled: true

tidb:
  tlsClient:
    enabled: true

That is all the whole API we needed.

If the user set tlsCluster.enabled to true, then tidb-operator will assume that there are several Secrets named with: <cluster-name>-pd-cluster-secret, <cluster-name>-tikv-cluster-secret, <cluster-name>-tidb-cluster-secret, <cluser-name>-cluster-client-secret and other TiDB components's Secrets are created.

Each Secret data should have three keys: tls.crt, tls.key and ca.crt:

apiVersion: v1
kind: Secret
metadata:
  name: <secret-name>
data:
  tls.crt: <base64 decoded certificate data>
  tls.key: <base64 decode key data>
  ca.crt: <base64 decode ca data>

These Secrets can be created by the use manually, by K8s builtin CA system or by cert-manager. PD/TiKV/TiDB/... will use these Secrets to start server.

tidb-operator will not supply these certificates automatically.

There are several tasks:

• Remove old auto generated codes
  • [x] remove TLS codes for client #1867
  • [x] remove TLS codes between components https://github.com/pingcap/tidb-operator/pull/1870
• TLS for MySQL Clients
  • [x] TiDB Server/Client #1867
  • [x] TiDB add require-secure-transport option to TiDB Server https://github.com/pingcap/tidb/pull/15341
  • [ ] TiDB Operator support require-secure-transport option
  • [x] TiDB Initializer https://github.com/pingcap/tidb-operator/pull/1931
  • [x] verify cert-based authentication manually
• TLS between TiDB components
  • [x] PD/TiKV/TiDB https://github.com/pingcap/tidb-operator/pull/1870
  • Pump/Drainer
  • [x] #1739
  • [x] #1779
  • [ ] Pump and Drainer always print http not https scheme, https://github.com/pingcap/tidb-binlog/issues/936
  • [x] TiDB-Binlog doesn't support wildcard SAN? https://github.com/pingcap/tidb-binlog/issues/935
  • [x] pd-ctl
  • [x] BR https://github.com/pingcap/tidb-operator/pull/1988
  • TiDB Monitor
  • [x] PD/TiDB #1917 #1919
  • [x] TiKV support TLS status API https://github.com/tikv/tikv/issues/5340 https://github.com/tikv/tikv/pull/5393
  • [ ] TiDB Operator supports TiKV TLS status API https://github.com/pingcap/tidb-operator/pull/2137
• Documents
  • TLS for MySQL Clients
  • user-defined TLS
    • [x] mysql-client
    • [x] TidbInitializer
  • K8s built-in certificate issue system
    • [ ] mysql-client
    • [ ] TidbInitializer
  • cert-manager issue system
    • [x] mysql-client
    • [x] TidbInitializer
  • TLS for TiDB Components
  • user-defined TLS
    • [x] PD/TiKV/TiDB/Monitor
    • [x] Pump/Drainer
    • [x] BR
  • K8s built-in certificate issue system
    • [x] PD/TiKV/TiDB/Monitor
    • [ ] Pump/Drainer
    • [ ] BR
  • cert-manager issue system
    • [x] PD/TiKV/TiDB/Monitor
    • [x] Pump/Drainer
    • [x] BR
• [ ] Test PD/TiDB/TiKV reload TLS certificate online
• Support allowed Common Name(CN)
  • [x] https://github.com/pingcap/pd/pull/2241
  • [x] https://github.com/pingcap/pd/pull/2251
  • [ ] PD support only one CN: https://github.com/pingcap/tidb-operator/issues/1998
  • [x] PD CN cert issue: https://github.com/pingcap/tidb-operator/issues/2002
  • [x] TiDB Should started with advertize-addr issue: https://github.com/pingcap/tidb-operator/issues/1859
  • [x] https://github.com/pingcap/tidb-binlog/pull/934
  • [x] https://github.com/pingcap/tidb/pull/15137
  • [x] https://github.com/tikv/tikv/pull/7134
  • [x] TiDB Operator supports cert-allowed-cn https://github.com/pingcap/tidb-operator/pull/2061
  • [x] can't connect to tidb status api: #2047
  • [x] failed to set pump cert-allowed-cn: https://github.com/pingcap/tidb-operator/issues/2046
  • [x] pd issue: https://github.com/pingcap/tidb-operator/issues/2048
• [ ] Add E2E test specs
• [ ] How to verify the Cluster/Client TLS is enabled correctly?
• [ ] Mydumper/Backup/BackupSchedule/Restore/DM(?) (after v1.1 GA)
• [ ] Lightning(https://github.com/pingcap/tidb-lightning/pull/270) and Importer(https://github.com/tikv/importer/pull/40) (after v1.1 GA)
• [ ] tidb probe problem: https://github.com/pingcap/tidb-operator/issues/2132 https://github.com/pingcap/tidb-operator/pull/2139
• [ ] cert-manager problem: https://github.com/pingcap/tidb-operator/issues/2127
• [ ] tidb-operator should use renew client cert: https://github.com/pingcap/tidb-operator/issues/2138

Low priority or do not do issues:

• mariadb-client problem (please use oracle mysql-client 5.7 instead)
  • tidb server failed to validity client certificate period
  • wrong client cert can connect to tidb server
• tikv-ctl https://github.com/tikv/tikv/issues/7016
• tidb-ctl https://github.com/pingcap/tidb/issues/15173

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 15 days