provider-server: send info of kernelMountOptions for cephfs to client when encryption in transit is enabled

[rohan47]

add kernel mount option ms_mode=secure to cephfs storageclass data when encryption in transit is enabled

[openshift-ci[bot]]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: rohan47 Once this PR has been reviewed and has the lgtm label, please assign jarrpa for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/red-hat-storage/ocs-operator/blob/main/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

[openshift-ci[bot]]

Hi @rohan47. Thanks for your PR.

I'm waiting for a red-hat-storage member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.

[rohan47]

Testing the code changes

[rohan47]

Tested the code changes

Csi config after the changes

oc get cm ceph-csi-configs -o yaml
apiVersion: v1
data:
  config.json: '[{"clusterID":"81fd58def37a35d1b4d4bee7ba35c3c3","storageClientID":"fb3e4f11-a38b-4859-ad18-1e8d579eb72b",
"monitors":["10.0.16.90:3300","10.0.61.203:3300","10.0.59.136:3300"],"rbd":{"radosNamespace":"cephradosnamespace-f6e94668817e4eefeb19b7a402fd947e"}},{"clusterID":"fe09c94a50c72d033da43478c21b8314",
"storageClientID":"fb3e4f11-a38b-4859-ad18-1e8d579eb72b","monitors":["10.0.16.90:3300","10.0.61.203:3300","10.0.59.136:3300"],
"cephFS":{"subvolumeGroup":"cephfilesystemsubvolumegroup-38beca789f5bc10e556b9832fd1070f6",
"kernelMountOptions":"ms_mode=secure"}}]'
kind: ConfigMap
metadata:
  creationTimestamp: "2024-07-22T19:24:10Z"
  name: ceph-csi-configs
  namespace: openshift-storage
  ownerReferences:
  - apiVersion: v1
    blockOwnerDeletion: true
    controller: true
    kind: ConfigMap
    name: ocs-client-operator-config
    uid: 664e804f-fc77-4aef-9387-a3c39dcb0b00
  resourceVersion: "3135818"
  uid: 7dee1733-974c-48f3-9cc7-7dbab17aee62

[agarwal-mudit]

/ok-to-test

[openshift-ci[bot]]

@rohan47: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/ocs-operator-bundle-e2e-aws	8ae40d62995bcd7fbfd342bbaef9f33e6b85b5c1	link	true	/test ocs-operator-bundle-e2e-aws

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).

[rohan47]

In the generated ceph config, shouldn't it also contain mons w/ 6789 port for ceph client to communicate w/o encryption?

In Case EiT is enabled on storagecluster all the clients are supposed to use 3300 right as mons will only listen on secure port?

[leelavg]

included in #2713 pls check & confirm.

[openshift-merge-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.