HDInsight cluster with OpenTSDB install failed

[AurelienGalicher]

During the deployment on azure, the installation procedure fails.

The custom script https://github.com/hdinsight/Iaas-Applications/tree/master/Scripts/install_ambari_app_bootstrap.sh fails to execute on the head node. the installation the stops and the edge node is not installed.

I am not able to log on the ambari console to check the log files.

[BeatGodglueck]

I found the same Problem. The error.txt and output.txt follow:

` --2018-05-23 15:57:03-- https://raw.githubusercontent.com/hdinsight/Iaas-Applications/master/Scripts/install_ambari_application.py Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.36.133 Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.36.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 23168 (23K) [text/plain] Saving to: ‘/tmp/install_ambari_application.py’

 0K .......... .......... ..                              100% 4.91M=0.005s

2018-05-23 15:57:03 (4.91 MB/s) - ‘/tmp/install_ambari_application.py’ saved [23168/23168]

--2018-05-23 15:57:03-- https://raw.githubusercontent.com/hdinsight/Iaas-Applications/master/Scripts/shared_ambari_installation.py Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.36.133 Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.36.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 16006 (16K) [text/plain] Saving to: ‘/tmp/shared_ambari_installation.py’

 0K .......... .....                                      100% 4.17M=0.004s

2018-05-23 15:57:03 (4.17 MB/s) - ‘/tmp/shared_ambari_installation.py’ saved [16006/16006]

hdinsight-datadisks 1.0 has requirement psutil<=5.0.1, but you'll have psutil 5.4.0 which is incompatible. hdinsight-kafka 1.1 has requirement psutil<=5.0.1, but you'll have psutil 5.4.0 which is incompatible. hdinsight-datadisks 1.0 has requirement psutil<=5.0.1, but you'll have psutil 5.4.0 which is incompatible. hdinsight-kafka 1.1 has requirement psutil<=5.0.1, but you'll have psutil 5.4.0 which is incompatible. Traceback (most recent call last): File "/tmp/install_ambari_application.py", line 275, in shared_lib.configure_loggers(args.logfile, args.verbosity) File "/tmp/shared_ambari_installation.py", line 30, in configure_loggers stdout_handler.addFilter(pip.utils.logging.MaxLevelFilter(logging.WARNING)) AttributeError: 'module' object has no attribute 'utils' Traceback (most recent call last): File "/var/lib/ambari-agent/cache/custom_actions/scripts/run_customscriptaction.py", line 194, in ExecuteScriptAction().execute() File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 306, in execute method(env) File "/var/lib/ambari-agent/cache/custom_actions/scripts/run_customscriptaction.py", line 179, in actionexecute ExecuteScriptAction.execute_bash_script(bash_script, scriptpath, scriptparams) File "/var/lib/ambari-agent/cache/custom_actions/scripts/run_customscriptaction.py", line 149, in execute_bash_script raise Exception("Execution of custom script failed with exit code",exitcode) Exception: ('Execution of custom script failed with exit code', 1) Downloading python scripts to /tmp PIP installing dependencies (purl & python-ambariclient) Collecting purl Downloading https://files.pythonhosted.org/packages/36/dc/252eedfe062c3f8b92b565ebbdfacbcf0e5522d2cd4c8873d9f88b096290/purl-1.4-py2.py3-none-any.whl Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from purl) (1.11.0) Installing collected packages: purl Successfully installed purl-1.4 Collecting python-ambariclient Downloading https://files.pythonhosted.org/packages/2a/4a/cb22b92da91daa9b6df27a25e0bfd8907915ee3c33706f9ef93643965a2a/python-ambariclient-0.6.0.tar.gz Requirement already satisfied: requests>=1.2.1 in /usr/local/lib/python2.7/dist-packages (from python-ambariclient) (2.9.1) Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from python-ambariclient) (1.11.0) Building wheels for collected packages: python-ambariclient Running setup.py bdist_wheel for python-ambariclient: started Running setup.py bdist_wheel for python-ambariclient: finished with status 'done' Stored in directory: /root/.cache/pip/wheels/3b/1b/9a/1414f3c1d5432af316765c2857fd801cc75bf854ddd2ed7e9d Successfully built python-ambariclient Installing collected packages: python-ambariclient Successfully installed python-ambariclient-0.6.0 Running python script to install Ambari service ('Start downloading script locally: ', u'https://raw.githubusercontent.com/hdinsight/Iaas-Applications/master/Scripts/install_ambari_app_bootstrap.sh') Fromdos line ending conversion successful ('Unexpected error:', "('Execution of custom script failed with exit code', 1)") Removing temp location of the script `

[Guido-Jacobs-MS]

I'm facing the same issue.

The whole process stops due the fact that the object "pip.utils.logging.MaxLevelFilter(logging.WARNING)" isn't found anymore. The script works perfectly in the past - I used it a lot! But now I have to find another way to create an OpenTSDB Cluster on-demand - sadly that nobody is looking after this repo anymore.

Error-Section: File "/tmp/shared_ambari_installation.py", line 30, in configure_loggers stdout_handler.addFilter(pip.utils.logging.MaxLevelFilter(logging.WARNING)) AttributeError: 'module' object has no attribute 'utils'

I created a pull request to the file 'shared_ambari_installation.py' and fixed the issue by myself. As long as this changes are not implemented into the master branch feel free to create a copy of the Files: install_ambari_app_bootstrap.sh shared_ambari_installation.py

Implement the suggested changes in the shared_ambari_installation.py file #20 and refer this new file in the install_ambari_app_bootstrap.sh script. Finally you have to edit the Azure ARM template file.

Adjust the URI in the scriptActions Section of the Headnode. "uri": "[concat('https://YOUR_PUBLIC_LOCATION/', 'install_ambari_app_bootstrap.sh')]",

Hope this helps :-)

[aaneja]

Merged #20. Thank you for your contribution!