Add encryption support for RBD volumes with IBM HPCS KMS

[humblec]

Describe the feature you'd like to have

IBM Cloud™ Hyper Protect Crypto Services is a key management and cloud hardware security module (HSM). It is designed to enable a user to take control of their cloud data encryption keys and cloud hardware security models, and is the only service in the industry built on FIPS 140-2 Level 4-certified hardware. The request here is to support this KMS integration thus enable the HPCS users to make use of the same in RBD volume operations.

Initial design/Identified changes:

Below parameters can be used to establish the connection to the HPCS service from the CSI driver and make use of the encryption operations:

KMS_SERVICE_NAME=[kms_service_name]
A unique name for the key management service within the project.

SERVICE_INSTANCE_ID=[service_instance_id]
The Instance ID of the IBM HPCS service, ex:  crn:v1:bluemix:public:hs-crypto:us-south:a/5d19cf8b82874c2dab37e397426fbc42:e2ae65ff-954b-453f-b0d7-fc5064c203ce::

SERVICE_API_KEY=[service_api_key]
Ex:  06x6DbTkVQ-qCRmq9cK-p9xOQpU2UwJMcdjnIDdr0g2R

CUSTOMER_ROOT_KEY=[customer_root_key]

BASE_URL=[base_url] → only required if the instance is in a different region
 The base URL (key management endpoint URL ) specifies the URL where your  HPCS instance resides. It is region specific. Ex:
https://api.us-south.hs-crypto.cloud.ibm.com:9756

TOKEN_URL=[token_url] → only required if the different than the default token url
 Ex:  https://iam.bluemix.net/oidc/token

IBM key protect APIs can be used for the integration and examples can be found here:

Doc:
https://cloud.ibm.com/docs/key-protect 

Clients:

https://github.com/IBM/keyprotect-go-client
https://github.com/IBM/keyprotect-python-client
https://github.com/IBM/keyprotect-java-client

Authentication:

https://github.com/IBM/keyprotect-go-client#authentication

Examples:

https://github.com/IBM/keyprotect-go-client#examples

afaict, Libopenstorage also got support for key protect APIs https://github.com/libopenstorage/secrets/tree/master/ibm , so we are good to use it via LOS or directlty the key protect clients

[humblec]

A couple of additional input here,

• [x] The key protect API Server and Client APIs is compatible and can work on HPCS or Key Protect KMS services. So, this feature can be developed for both at the same time with minor changes.
• [ ] The LibOpenstorage api interation looks like bit old (2 years) https://github.com/libopenstorage/secrets/tree/master/ibm , so one thing which we need to take a decision here is : what is the preferred way of integration here ? ie, will use libopenstorage or use direct client SDKs of key protect APIs? @nixpanic any preference ?

[nixpanic]

I do not have a preference over using libopenstorage or the KeyProtect client directly. If Rook uses libopenstorage, I'd got with that too.

The Amazon KMS provider does not use libopenstorage either.

[humblec]

One other option we have here is exploring KMIP apis for the integration side. exploring those options as well, before we start the implementation here. We can not do both at the same time or in single phase so have to weigh one against other and go ahead accordingly.

[humblec]

After some disussions on this, we are going ahead with key protect API SDK for this inegration atleast in the first phase.

[humblec]

* [ ]  The LibOpenstorage api interation looks like bit old (2 years) https://github.com/libopenstorage/secrets/tree/master/ibm , so one thing which we need to take a decision here is :   what is the preferred way of integration here ? ie, will use libopenstorage or use direct client SDKs of key protect APIs? @nixpanic  any preference ?

The libopenstorage client protect API update PR is in up for review https://github.com/libopenstorage/secrets/pull/58 :+1:

[humblec]

Unfortunately the LBS repo maintenance looks to be really slow/not progressing. We will use KP client SDK directly for this integration.

[humblec]

This has been done and PR is merged, one small update to the internal code base about the env variable name is waiting for confirmation from other related components for consistency. As soon as its available, will clear the milestone or close this issue.

[humblec]

This has been done and PR is merged, one small update to the internal code base about the env variable name is waiting for confirmation from other related components for consistency. As soon as its available, will clear the milestone or close this issue.

https://github.com/ceph/ceph-csi/pull/2747 should address this.

[humblec]

Removing from the release tracker, however keeping this issue open.

[Madhu-1]

@humblec what else is pending here?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs. Thank you for your contributions.

[Madhu-1]

@humblec what else is pending here?

[humblec]

Verification from cloud provider is still going on, based on the feedback I have to adjust the requirement or call it as Done.

[humblec]

As I havent heard any issues so far on this, I am closing this one.. will reopen if required..