search

hortonworks / ansible-hortonworks

Ansible playbooks for deploying Hortonworks Data Platform and DataFlow using Ambari Blueprints
Apache License 2.0
248 stars 253 forks source link

Ambari-2.7.4 incompatible with HDF mpacks (tested on HDF 3.4.1.1, 3.4.0) #166

Closed lhoss closed 5 years ago

lhoss commented 5 years ago

Hi @alexandruanghel , Not sure if you tested a cluster install with the latest Ambari (active since your latest version bump commit) and an HDF (mpack) deployment, but it seems those mpacks are not (yet) compatible with Ambari 2.7.4, because they are missing a file for only that version (smartsense-ambari-view-2.7.4.jar, details below):

Here the full stacktrace (when I reproduced the mpack install issue, outside ansible, in the shell, as user root):

ambari-server install-mpack --mpack=http://public-repo-1.hortonworks.com/HDF/centos7/3.x/updates/3.4.1.1/tars/hdf_ambari_mp/hdf-ambari-mpack-3.4.1.1-4.tar.gz -v

INFO: Management pack hdf-ambari-mpack-3.4.1.1-4 successfully installed! Please restart ambari-server.
INFO: Executing after-install hook script : /var/lib/ambari-server/resources/mpacks/hdf-ambari-mpack-3.4.1.1-4/hooks/after_install.py
INFO: about to run command: ['/usr/bin/ambari-python-wrap', u'/var/lib/ambari-server/resources/mpacks/hdf-ambari-mpack-3.4.1.1-4/hooks/after_install.py']
INFO:
process_pid=24065
ERROR: Failed to execute after-install hook. Failed with error code after-install
ERROR: Traceback (most recent call last):
  File "/var/lib/ambari-server/resources/mpacks/hdf-ambari-mpack-3.4.1.1-4/hooks/after_install.py", line 36, in <module>
    smartsense_versioner.fix_smartsense_versions()
  File "/var/lib/ambari-server/resources/mpacks/hdf-ambari-mpack-3.4.1.1-4/hooks/smartsense_versioner.py", line 58, in fix_smartsense_versions
    shell.checked_call(["cp", "-f", source_view_jar_file_path, new_view_jar_file_path], sudo=True)
  File "/usr/lib/ambari-server/lib/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/ambari-server/lib/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns)
  File "/usr/lib/ambari-server/lib/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/ambari-server/lib/resource_management/core/shell.py", line 314, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'cp -f /var/lib/ambari-server/resources/stacks/HDF/3.2.b/services/SMARTSENSE/package/files/view/smartsense-ambari-view-2.7.4.jar /var/lib/ambari-server/resources/stacks/HDF/3.2.b/services/SMARTSENSE/package/files/view/smartsense-ambari-view-1.5.0.2.7.4.0-118.jar' returned 1. cp: cannot stat '/var/lib/ambari-server/resources/stacks/HDF/3.2.b/services/SMARTSENSE/package/files/view/smartsense-ambari-view-2.7.4.jar': No such file or directory

ERROR: Exiting with exit code -1.
REASON: Failed to execute after-install hook. Failed with error code after-install

I thought maybe I got bad timing, with the currently active HDF build (4), but the same issue when testing with HDF-3.4.0.0

alexandruanghel commented 5 years ago

I know I've seen that but nothing I can do... this is a problem with Ambari and/or HDF mpack rather than an issue with these playbooks.

lhoss commented 5 years ago

I know I've seen that but nothing I can do.

@alexandruanghel yeah I'm totally aware that it's not a bug in your/our playbooks 👍 I should have mentioned that, and ask if you could ask inside Cloudera about the status of that issue, and ideally post here a reference/link so we can track the progress (if it's not tracked internal only) Also imagine any (new) user that now run the deployment with the bumped version (and HDF included), it will fail and they might look for help (hopefully checking the Issues section here)

lhoss commented 5 years ago

@alexandruanghel I just did another test run, with the hope that a newer build would have been released in the meantime, to fix this major issue , but it seems still not fixed! Would be great if you could ask inside Cloudera if this issue is known, and share the related AMBARI Jira issue if there's one?!

alexandruanghel commented 5 years ago

Don't think there will be a jira, probably non essential work on Ambari has stopped.

Looking at https://supportmatrix.hortonworks.com/ I can see it was updated to reflect the fact that Ambari 2.7.4 is not compatible with any HDF releases. So I guess that settles it, sorry.

lhoss commented 5 years ago

Thanks for the update I also just found the related community post: https://community.cloudera.com/t5/Support-Questions/Latest-HDP-3-1-4-not-compatible-with-any-version-of-HDF-or/m-p/277721/highlight/false#M207654 .. and had posted a summary of my learnings, with a link back to the issue here