apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.31k stars 1.24k forks source link

Support seamless TLS certificate rotation #12107

Open mgranderath opened 8 months ago

mgranderath commented 8 months ago

In our setup we use regularly rotated TLS certificates by essentially replacing the keystore/truststore (using K8s secrets) but the updated certificates don't get picked up by the nodes. This means that our nodes get restarted because the liveness probe starts failing which has some unintended consequences. Ideally we would want these to be picked up seamlessly.

Possible Solution

Wrapping the KeyManager and TrustManager and check every x interval whether the underlying files have been updated and replace the delegate. An example of an implementation somewhat similar is here

Jackie-Jiang commented 8 months ago

cc @xiangfu0 @zhtaoxiang

mgranderath commented 8 months ago

I can take a stab at this as well and upstream the changes if that would be helpful?

zhtaoxiang commented 6 months ago

Hi @mgranderath , we have similar issues as you described here.

I am working on PRs to use the sslcontext-kickstart to make the KeyManager and TrustManager swappable. Will post thoes PRs when it's ready for review

zhtaoxiang commented 5 months ago

I will keep the following PR list updated: Merged PRs: https://github.com/apache/pinot/pull/12277 https://github.com/apache/pinot/pull/12325 https://github.com/apache/pinot/pull/12357 https://github.com/apache/pinot/pull/12384 https://github.com/apache/pinot/pull/12404 https://github.com/apache/pinot/pull/12425 https://github.com/apache/pinot/pull/12455

Working in progress: https://github.com/apache/pinot/pull/12462