[feature] parse all JSON credential content into plain secret

[cdlliuy]

When playing with Cloud object storage service, its credential looks like:

{
  "apikey": "xxxxx",
  "cos_hmac_keys": {
    "access_key_id": "xxxx",
    "secret_access_key": "xxxx"
  },
  "endpoints": "https://control.cloud-object-storage.cloud.ibm.com/v2/endpoints",
  "iam_apikey_description": "xxxx",
  "iam_apikey_name": "test-cos-crendentail",
  "iam_role_crn": "xxxx",
  "iam_serviceid_crn": "xxxx",
  "resource_instance_id": "xxxx"
}

In current implementation, the cos_hmac_keys are saved as the JSON format in kube secret.

apiVersion: v1
data:
  apikey: xxx
  cos_hmac_keys: xxx
  ....
kind: Secret

Is that possible to parse the inner JSON as well to create secret as below format ?

apiVersion: v1
data:
  apikey: xxx
  cos_hmac_keys.access_key_id: xxx
 cos_hmac_keys.secret_access_key: xxx
  ....
kind: Secret

[ZhuangYuZY]

Discussion in https://ibm-cloudplatform.slack.com/archives/CU3795MNE/p1587454501008000

[ZhuangYuZY]

From Mandana Vaziri and @pdettori

The user will be able to specify what they want in the secret explicitly (optionally) by having the following in the spec of the binding resource:

secret-content:
- name: cos-hmac-keys-access-key-id
  value: <path-to-value-in-credentials>
- name: cos-hmac-keys-secret-access-key
  value: <path-to-value-in-credentials>

where the paths will be given using go templates. This would be accomodate other use cases we’ve seen where the credentials are very complicated and that also need some data transformation. Our Binding controller will take this spec and produce the correct data in the secret

[ZhuangYuZY]

My response: Thinking from end user perspective, ICO used for Coligo project, Coligo will expose the CLI to create binding CR to get credential from cloud to create secret in kube. The CLI will be like coligo create binding --service-instance service1 --credential-name credential1 From CLI perspective, it is hard to know how to create secret-content , to do that it will need to parser the credential json to see if it has 2 levels json, then create the template. I think it is not the responsiblity for CLI, but for ICO. Because ICO is special for IBM Cloud service integration it should service credential parser, and ICO already parser the service credential json (first level). So suggest to add new field in binding CR spec or annotation to indicate to parser all json levels to generate plain secret, like recursive-parser: true It will help to keep back compatible, and also support service credential which has >2 json levels.

[vazirim]

@Zhuang The coligo CLI will surely have to accept payloads for parameters. The above command works for simple cases, but in general bindings have parameters (to set the level of permissions for example). The shape of the secret created can be passed in the CLI in the same way.

A 2-level json unwinding is not enough for credentials of e.g. postgresql, which is many levels deep, and has arrays.

[ZhuangYuZY]

@vazirim Thank you for your response. And you provided an example for postgres which has multiple json levels of credential as below.

{
  "connection": {
    "cli": {
      "arguments": [
        [
          "host=95346614-149f-408a-afe8-<...>.databases.appdomain.cloud port=30361 dbname=ibmclouddb user=ibm_cloud_<...> sslmode=verify-full"
        ]
      ],
      "bin": "psql",
      "certificate": {
        "certificate_base64": "<...>"
      },
      "composed": [
        "PGPASSWORD=<...> PGSSLROOTCERT=<...>  psql 'host=95346614-149f-408a-afe8-<...>.databases.appdomain.cloud port=30361 dbname=ibmclouddb user=ibm_cloud_<...> sslmode=verify-full'"
      ],
      "environment": {
        "PGPASSWORD": "<...>",
        "PGSSLROOTCERT": "<...>"
      },
      "type": "cli"
    },
    "postgres": {
      "authentication": {
        "method": "direct",
        "password": "<...>",
        "username": "ibm_cloud_<...>"
      },
      "certificate": {
        "certificate_base64": "<...> f"
      },
      "composed": [
        "postgres://ibm_cloud_393ebef5_1225_4834_8785_<...>@95346614-149f-408a-afe8-<...>.databases.appdomain.cloud:30361/ibmclouddb?sslmode=verify-full"
      ],
      "database": "ibmclouddb",
      "hosts": [
        {
          "hostname": "95346614-149f-408a-afe8-<...>.databases.appdomain.cloud",
          "port": 30361
        }
      ],
      "path": "/ibmclouddb",
      "query_options": {
        "sslmode": "verify-full"
      },
      "scheme": "postgres",
      "type": "uri"
    }
  },
  "instance_administration_api": {
    "deployment_id": "crn:v1:bluemix:public:databases-for-postgresql:us-south:a/<...>:95346614-149f-408a-afe8-00099dd89082::",
    "instance_id": "crn:v1:bluemix:public:databases-for-postgresql:us-south:a/<...>:95346614-149f-408a-afe8-00099dd89082::",
    "root": "https://api.us-south.databases.cloud.ibm.com/v4/ibm"
  }
}

I agree that it is hard to do plane format conversion. So I think your proposal above is good to help end user to parser these multiple level json format as end user may only care of several parameters but not all of them.

Thank you.