Ambari 2.7 and HDP 3.0 - Stack HDP 3.0 is not found in Ambari metainfo

[markokole]

Here are my initial values:

cluster_name: 'hdp3'
ambari_version: '2.7.0.0'                                 # must be the 4-part full version number
hdp_version: '3.0.0.0'                                    # must be the 4-part full version number
hdp_build_number: '1634'                                  # the HDP build number from docs.hortonworks.com (if set to 'auto', Ansible will try to get it from the repository)

Here is the error message:

An internal system exception occurred: Stack data, Stack HDP 3.0 is not found in Ambari metainfo

It fails on this task: TASK [ambari-config : Register the VDF with Ambari (Ambari >= 2.6)] ****

I see Ambari 2.6.2.2 is installed This worked some weeks ago, did I miss something?

[alexandruanghel]

Hmm, strange, you should definitely have Ambari 2.7 installed for HDP3.0, so the fact that Ambari 2.6.2.2 was installed makes me believe you first ran the playbooks with ambari_version: '2.6.2.2' and then switched to 2.7.0.0.

There is a small issue with how the yum_repository Ansible module works in that it adds new repos rather than replace them. So check your /etc/yum.repos.d/ambari.repo file, if it contains references for both Ambari 2.6 and 2.7 then it means the above happened so you'll need to rebuild your nodes or manually upgrade Ambari to 2.7 before installing HDP 3.0.

The only other change that happened was that Ambari 2.7.1 and HDP 3.0.1 were released on Friday, but that shouldn't have impacted the Ambari 2.7.0 repos.

[markokole]

I see it now. I forgot to add --extra-vars and my hdp-config file in install_ambari step and I guess it takes version 2.6.2.2 as default. If so, is it smart to have an Ambari version default?

[lhoss]

(I wanted to propose following before your last answer) btw @markokole HDP 3.0.1 is out (and I guess recommended vs 3.0.0), and I just made a successful test deployment on AWS, using following variable changes:

#ambari_version: '2.7.0.0'                                 # must be the 4-part full version number
#hdp_version: '3.0.0.0'                                    # must be the 4-part full version number
#hdf_version: '3.2.0.0'                                    # must be the 4-part full version number

ambari_version: '2.7.1.0'                                 # must be the 4-part full version number
hdp_version: '3.0.1.0'                                    # must be the 4-part full version number
hdf_version: '3.2.0.0'                                    # must be the 4-part full version number

@alexandruanghel should I create a PR to update the new default version to 3.0.1 ? ( ps: I'm planning on creating an issue, to make the roles+playbooks work when called from an external repository/folder, which allows us to manage our customer/cluster configs in a separate github repo )

[alexandruanghel]

@markokole not sure I understand, if you use --extra-vars you'll override any variable so you have to use the same --extra-vars everywhere. Or better, use the install_cluster script rather than the individual scripts. The --extra-vars option should be used very rarely (when testing something for example or to override a simple variable) and you should only rely on group_vars/all for everything else and maintain it. Some configs depend on one another and only group_vars/all will offer the full picture of what is installed and should be running (I'm using here the "One Mona Lisa" principle as recommended by Ansible).

@lhoss I'll update the versions later today as I'm testing some combinations. I was waiting for the first maintenance release to upgrade more defaults (for example if the default hdp version change then the default blueprint_dynamic variable will also need to change).

And yes, please do create issues and share ideas as I can't be aware of how people are using this :)

[markokole]

@alexandruanghel : on this project, we are using this repository, but we are not allowed to make any changes in the code from repository - im manipulating everything using --inventory and --extra-vars. So far it works :) Now Im trying to figure out how to create an HDP cluster but NOT start the services in Ambari.

I have a small demo for AWS here: https://github.com/markokole/ansible-hortonworks-with-extra-vars And another demo for static here (this one is using terraform with consul) : https://github.com/markokole/aws-consul-terraform

[markokole]

@lhoss

(I wanted to propose following before your last answer) btw @markokole HDP 3.0.1 is out (and I guess recommended vs 3.0.0), and I just made a successful test deployment on AWS, using following variable changes:

#ambari_version: '2.7.0.0'                                 # must be the 4-part full version number
#hdp_version: '3.0.0.0'                                    # must be the 4-part full version number
#hdf_version: '3.2.0.0'                                    # must be the 4-part full version number

ambari_version: '2.7.1.0'                                 # must be the 4-part full version number
hdp_version: '3.0.1.0'                                    # must be the 4-part full version number
hdf_version: '3.2.0.0'                                    # must be the 4-part full version number

@alexandruanghel should I create a PR to update the new default version to 3.0.1 ? ( ps: I'm planning on creating an issue, to make the roles+playbooks work when called from an external repository/folder, which allows us to manage our customer/cluster configs in a separate github repo )

What build_number did you specify for 3.0.1? Im asking this because for 3.0.0 I had to write 1634 for HDP build_number. Thanks

[lhoss]

What build_number did you specify for 3.0.1? Im asking this because for 3.0.0 I had to write 1634 for HDP build_number.

@markokole In fact I used the defaults:

hdp_build_number: 'auto'                                  # the HDP build number from docs.hortonworks.com (if set to 'auto', Ansible will try to get it from the repository)
hdf_build_number: 'auto'                                

This worked for me for both HDP versions: 3.0.1.0 and 3.0.0.0 ! Actually I just saw the latest commit in master, using the latest versions: https://github.com/hortonworks/ansible-hortonworks/commit/9bf56742650e03e7762bbe559d497bbbeba89529 and also still the default build_numbers