search

ibm-watson-data-lab / ibmos2spark

Facilitates Data I/O between Spark and IBM Object Storage services.
10 stars 8 forks source link

CloudObjectStorage should remove bluemix_cos and soflayer_cos #46

Open gilv opened 6 years ago

gilv commented 6 years ago

current function pattern is : CloudObjectStorage(sc, credentials, configurationName, "bluemix_cos") It's wrong to call "bluemix_cos" for IAM usage and SoftLayer_cos for usage with access key and password. Both brandings are not in use by IBM anymore. It also confuse our customers and i suggest the last entry in the function better be removed at all.

I suggest: If user provided credentials with api_key, service_jd then use IAM based authentication, otherwise if user provided access_key then use old style authentication. If user provided both - then choose IAM as default.

bassel-zeidan commented 6 years ago

bluemix_cos and SoftLayer_cos are the official names for Cloud Object Storage different deployments on Softlayer and on Bluemix. Using IAM or something else is just a technical term related to the auth process used for a specific model and this might change. In fact, Bluemix cos now supports HMAC credentials as well.

We can't just rely on the passed attributes because those attributes might be updated and changed and one service might support multiple auth systems which make it harder to detect the target model. make sense?

I feel the current solution is cleaner. However, Adam and I wanted to do some updates on the CloudObjectStorage internal logic to get rid of the excessive use of if-else checks. We will follow up on that soon.