Trying to install CDP private cloud, but still need AWS credentials

[gshilo]

Tried to install CDP private cloud. I updated the inventory file to include all my hosts, but during the deployment it still tries to connect to AWS and complains it does not have the appropriate credentials:

TASK [cloudera_deploy : Get AWS Account Info] **** Thursday 20 May 2021 22:45:31 +0000 (0:00:00.036) 0:00:38.058 ** An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.NoCredentialsError: Unable to locate credentials fatal: [localhost]: FAILED! => {"boto3_version": "1.17.66", "botocore_version": "1.20.66", "changed": false, "msg": "Failed to retrieve caller identity: Unable to locate credentials"}

How do I tell it this is a private cloud installation that has nothing to do with AWS ?

[Chaffelson]

yep, good find, I think I have a workaround for you. Insert this between line 57 and 58 of main.yml you can add this to the list of 'when' statements: use_download_mirror | default(default_enable_download_mirror) | bool

If that solves your issue can you please let me know

[gshilo]

This doesn't seem to work... I changed main.yml as you suggested, but at runtime it gave me the same error mrssage:

TASK [cloudera_deploy : Get AWS Account Info] **** Saturday 22 May 2021 21:45:56 +0000 (0:00:00.022) 0:00:32.962 ** An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.NoCredentialsError: Unable to locate credentials fatal: [localhost]: FAILED! => {"boto3_version": "1.17.66", "botocore_version": "1.20.66", "changed": false, "msg": "Failed to retrieve caller identity: Unable to locate credentials"}

There is also main.tml in cloudera_deploy/roles/cloudera_deploy/defaults. There are those lines in it:

Default Deployment Controls default_infra_type: aws default_infra_region: us-east-1

I believe this is the reason it looks for AWS credentials. However, I do not know what infra_type I should use in order to install on private cloud.

Thanks

[Chaffelson]

main.yml is just the default filename that Ansible looks for, it can occur in a lot of directories.

If you can give me the details of the definition and profile you are using, and the command you used to execute the run, I can try to reproduce today. I'll try to repro on my local machine anyway with a default run, but it's best if I have your details

[Chaffelson]

Reproduced and fixed on my local machine. Take a look at the changes in the commit and see if they work for you, if they do I'll get @wmudge to include this as a hotfix.

[wmudge]

Any update on this? We are going to cut a release very soon.

[AlexHff]

Hello everyone, I am having the same issue as described above. After implementing the changes submitted by @Chaffelson, the first error does not appear anymore. But this doesn't fix the issue. I now have another error

TASK [cloudera.exe.infrastructure : Query AWS VPCs by unique name and CIDR] ******************************************
Thursday 03 June 2021  20:58:52 +0000 (0:00:00.075)       0:00:09.332 ********* 
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.N
oCredentialsError: Unable to locate credentials
fatal: [localhost]: FAILED! => {"boto3_version": "1.17.86", "botocore_version": "1.20.86", "changed": false, "msg": "U
nable to describe VPCs []: Unable to locate credentials"}

Here is some more context: I have defined an inventory file and am using the following command: ansible-playbook /opt/cloudera-deploy/main.yml -e "definition_path=examples/sandbox" -i /opt/cloudera-deploy/inventory_static.ini -t run,default_cluster. Everything else is default.

In the end, what fixed the problem was changing the tags from run,default_cluster to full_cluster.

[Chaffelson]

@AlexHff thanks for the detailed description of your steps and the error you are getting.

I suspect you have not made any modifications inexamples/sandbox/definition.yml, and by default that defines a CDP Public deployment with a Kafka cluster, and a CDP Private Base cluster. These definitions are activated by the tags passed in, and by passing in 'run' you are instructing cloudera-deploy to execute the CDP Public part of the deployment definition. If you just use default_cluster or full_cluster you are only activating the CDP Private Base part of the definition.

So in essence, the directory contains the Definition, which is composed together with the user profile in ~/.config/cloudera-deploy/profiles. The Ansible Tags (run,default_cluster) specify the Goals for that deployment execution, and then the playbook goes and does its business.

So, your solution is a correct one and the easiest, but you could also create a custom Definition which only describes your specific requirements, and pass that directory instead of examples/sandbox. Hopefully that all makes sense, more detailed documentation around all this is forthcoming.

[Chaffelson]

No further activity on this ticket, closing pending more reports of the issue.