Open Tema opened 2 years ago
@Tema Thanks for reporting this issue! The request is reasonable, would you like to contribute to this feature? BTW, with "Or allow security/cacert-path to take precedence of secrete instead of blindly overriding it", how do you want to do this exactly? And could you please describe the update to the CR briefly so that we can reach agreement before coding, thanks!
"Or allow security/cacert-path to take precedence of secrete instead of blindly overriding it", how do you want to do this exactly?
Basically instead of https://github.com/pingcap/tidb-operator/blob/62b11ad37ddd3482dfec519770d6c65011e24d6d/pkg/manager/member/pd_member_manager.go#L836-L841
I thought we can have
if tc.IsTLSClusterEnabled() {
if config.Get("security.cacert-path") == nil {
config.Set("security.cacert-path", path.Join(pdClusterCertPath, tlsSecretRootCAKey))
}
}
The only caveat is if someone already have this defined with some value, which I would say highly unlikely it might break things. Maybe you can put it in the next release note to avoid this chance.
Alternatively, we can always mount -root-ca-secret
with optional flag and take path.Join(pdClusterCertPath, tlsSecretRootCAKey)
in the snippet above from it if the secret is present.
Is it possible to accomplish your needs by enhancing the additionalVolumeMount
configuration, if you configure the same volume name, it will use what you configured. This is similar to the idea of 3350. @Tema @DanielZhangQD
@mikechengwei
Is it possible to accomplish your needs by enhancing the additionalVolumeMount configuration, if you configure the same volume name
I've tried that, but k8s forbids mounting to the same name multiple entities and <cluster-name>-pd-cluster-secret
is already mounted to the /var/lib/pd-tls
. Subpath mounting faces the same issue.
The only caveat is if someone already have this defined with some value, which I would say highly unlikely it might break things. Maybe you can put it in the next release note to avoid this chance.
Agree with you.
So we can go with the below solution, right?
additionalVolume
with the separate ca secret, the additionalVolumeMount
, and security.cacert-path
Sounds good @DanielZhangQD.
@Tema Would you like to submit PR and an example yaml in the examples
dir for this?
@DanielZhangQD, we have decided to use TiDB layer in our setup. Fortunately, tidb-operator allows to specify separate certificates for client and inter-component already, so it does not block us anymore. However, in future I believe we will still want to have this feature to be able to use Raw TiKV directly. Just not right now, hence I'm deferring work on this PR.
@Tema now,you can try this feature , the same volume name in additionalVolumeMount it will overwrite default volume.
Feature Request
Is your feature request related to a problem? Please describe: We use tidb-operator to run TiKV cluster w/o TiDB layer. Our clients are deployed to a separate k8s cluster. It is not safe to share private key of root CA, so our clients have a separate root CA. Now I have to provider PD/TiKV with combined ca.crt from client and server. We rely on cert-manager.io to generate and rotate certificates as recommended on https://docs.pingcap.com/tidb-in-kubernetes/stable/enable-tls-between-components#using-cert-manager, but it does not allow to add an additional
ca.crt
in the generated secrets.Describe the feature you'd like: Allow to specify a separate secret for root CA to override
ca.crt
from-pd-cluster-secret
. Or allowsecurity/cacert-path
to take precedence of secrete instead of blindly overriding it in https://github.com/pingcap/tidb-operator/blob/62b11ad37ddd3482dfec519770d6c65011e24d6d/pkg/manager/member/pd_member_manager.go#L836-L841 so that it is possible to mount and configure it separatelyDescribe alternatives you've considered:
-pd-cluster-secret
mount.additionalVolumeMount
in CR withsubPath
to mount a different ca.crt under/var/lib/pd-tls
, but k8s forbids mounting to the same path/etc/ssl/certs/ca-certificates.crt
on the pod (system trust store), but PD and TiKV doesn't seem honoring it.ca.crt
into secret for the cert-manager.ioIssuer
, but cert-mahager.ioCertificate
does not include it in the generated certificate secrete. The only alternative I can see is not using cert-manager.io, but generate secrete manually, however then I will have to implement in infrastructure to rotate certificates.Teachability, Documentation, Adoption, Migration Strategy: