get_credentials generates exception

[gilv]

The following code

`import ibm_boto3

if name == 'main': session = ibm_boto3.Session() credentials_session = session.get_credentials() `

Generates

Traceback (most recent call last): File "/Users/aa/Dev/sky/examples/general/pytest1.py", line 5, in credentials_session = session.get_credentials() File "/Users/aa/.pyenv/versions/3.9.1/lib/python3.9/site-packages/ibm_boto3/session.py", line 238, in get_credentials return self._session.get_credentials() File "/Users/aa/.pyenv/versions/3.9.1/lib/python3.9/site-packages/ibm_botocore/session.py", line 488, in get_credentials self._credentials = self._components.get_component( File "/Users/aa/.pyenv/versions/3.9.1/lib/python3.9/site-packages/ibm_botocore/credentials.py", line 2076, in load_credentials creds = provider.load() File "/Users/aa/.pyenv/versions/3.9.1/lib/python3.9/site-packages/ibm_botocore/credentials.py", line 1277, in load available_creds = self._ini_parser(self._creds_filename) File "/Users/aa/.pyenv/versions/3.9.1/lib/python3.9/site-packages/ibm_botocore/credentials.py", line 2267, in load_ibm_credentials_filename _data = self.get_data(path) File "/Users/aa/.pyenv/versions/3.9.1/lib/python3.9/site-packages/ibm_botocore/credentials.py", line 2260, in get_data return json.load(f) File "/Users/aa/.pyenv/versions/3.9.1/lib/python3.9/json/init.py", line 293, in load return loads(fp.read(), File "/Users/aa/.pyenv/versions/3.9.1/lib/python3.9/json/init.py", line 346, in loads return _default_decoder.decode(s) File "/Users/aa/.pyenv/versions/3.9.1/lib/python3.9/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/Users/aa/.pyenv/versions/3.9.1/lib/python3.9/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)

[gilv]

What location it looks for configuration file?

[IBMalok]

@gilv - configuration location will be - ~/.bluemix/cos_credentials For more details, you can refer - https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-python#python-credentials

[gilv]

@IBMalok then why it fails? I have this file there with

[default] aws_access_key_id = aa aws_secret_access_key = bb

[gilv]

@IBMalok by the way, ibmcloud cos plugin, will create configuration file in ~/.bluemix/plugins/cloud-object-storage/config.json

[gilv]

@IBMalok If I take aws config from ~/.aws that works with boto3 and copy-paste it as ~/.bluemix/cos_credentials then it will also fails with ibm_boto3

[pavelexpertov]

Try exporting COS_CREDENTIALS_FILE=~/.bluemix/cos_credential as an environment variable (point it to a different path if you must) before running your code. Whilst debugging, it seems to make the botocore pick up the config file. Despite the fact it is a default value for config file location in the first place.

[gilv]

@pavelexpertov same stuff...I take valid ~/aws/credentials that works with boto3, copy paste it into ~/.bluemix/cos_credentials and read it with ibm_boto3 (setup the env variable as you suggested ) and getting File "/Users/gilvernik/.pyenv/versions/3.9.1/lib/python3.9/site-packages/ibm_botocore/credentials.py", line 2260, in get_data return json.load(f) File "/Users/gilvernik/.pyenv/versions/3.9.1/lib/python3.9/json/__init__.py", line 293, in load return loads(fp.read(), File "/Users/gilvernik/.pyenv/versions/3.9.1/lib/python3.9/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/Users/gilvernik/.pyenv/versions/3.9.1/lib/python3.9/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/Users/gilvernik/.pyenv/versions/3.9.1/lib/python3.9/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 2 (char 1)

[gilv]

@pavelexpertov I also want to add iam key to the properties file, is this correct? [default] aws_access_key_id = aa aws_secret_access_key = bb ibm_api_key_id = cc

[pavelexpertov]

No. cos_credentials file has to be in a json format (as indicated in the json decode error). Try exporting the environment variable of AWS_CONFIG_FILE and point it to the aws file. As for how the config should look like, you should refer it to the docs, because I haven't used it for my use case.

[pavelexpertov]

Refer to this file to see where the environment variables are at.

[gilv]

@pavelexpertov thanks, but I am not sure I understand. My cos_credentials is wrong? If I follow https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-python then it suppose to be the same as I am using.

[pavelexpertov]

Read carefully: "If the cos_credentials file contains HMAC keys the client authenticates with a signature, otherwise the client uses the provided API key to authenticate by using a bearer token (using an API key still requires the config=Config(signature_version="oauth") to be included during client creation)."

It shows example of aws config file, but not this one (when you generate credentials for the hmac or api keys on IBM cloud dashboard):

{
  "apikey": "",
  "cos_hmac_keys": {
    "access_key_id": "",
    "secret_access_key": ""
  },
  "endpoints": ""
  "iam_apikey_description": "",
  "iam_apikey_name": "",
  "iam_role_crn": "",
  "iam_serviceid_crn": "",
  "resource_instance_id": ""
}

By the way, the format, in terms of key names is possibly wrong because i don't have the original file at hand.

[pavelexpertov]

@IBMalok honestly, somebody whose responsibility to write documentation for authentication files (including the json credentials that get generated from the ibm cloud dashboard) should re-write this to increase examples of configuration files and ~the fact that they should export environment variables to get the library to work~ (so I found out that the library works, but it expects the json file to have specific key names in order to pull correct credentials. You will need to expand on json configuration file example since it's not outlined anywhere). It is kinda a terrible dev experience when one has to look through your dependencies to understand how on earth it reads credentials.

[gilv]

@pavelexpertov @IBMalok thanks

[avinash1IBM]

Closing as resolves