Starting a new thread with the installation problem

[kundeng]

Hi, just want to start a new thread to document all the steps I took and failed. I hope you could create a video or something which would benefit a lot of people.

1. Install using this guide.

https://cwiki.apache.org/confluence/display/AMBARI/Quick+Start+Guide

• setup hosts file

• adjust memory and cpus in Vagrant file

• setup just 1 node. in cento7.4 directory, run ./up.sh 1

• use vagrant snapshot to save state!

2. install ambari server

wget -O /etc/yum.repos.d/ambari.repo http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/ambari.repo && yum --enablerepo=extras install epel-release -y && yum install nano java java-devel mariadb mariadb-server mysql-connector-java ambari-server ambari-agent -y && ambari-server setup -s && ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar && ambari-server start && ambari-agent start && service mariadb start && chkconfig mariadb on

3. setup the cluster

I initially selected HDFS YARN and MAPreduce2 HIVe HBASE OOZI ZooKeeper, Ambari Metrics and Spark2

notes 1. I also selected all spark related client components livy and thrift.

notes 2. the installation took a long time and it also used a lot of memory. I am not sure how to tune it. Sorry, new to HDP.

4. setup HUE

I ran python /var/lib/ambari-server/resources/scripts/configs.py -u admin -p admin -n [CLUSTER_NAME] -l [CLUSTER_FQDN] -t 8080 -a set -c cluster-env -k ignore_groupsusers_create -v true and [CLUSTER_NAME] is the name given to the cluster and [CLUSTER_FQDN] is c7401.ambari.apache.org

I ran sudo git clone https://github.com/steven-dfheinz/HDP3-Hue-Service.git /var/lib/ambari-server/resources/stacks/HDP/3.1/services/HUE

service ambari-server restart

5. issues

After that I tried to install HUE from the GUI and failed with errors.

I ran this to reset, and tried again. Still no luck.
rm -rf /var/lib/ambari-server/resources/stacks/HDP/3.1/services/HUE rm -rf /var/lib/ambari-agent/cache/stacks/HDP/3.1/services/HUE/ rm -rf /usr/local/hue rm -rf /usr/hdp/current/hue-server rm -rf /usr/local/hue-3.11.0/

[steven-matison]

This is pretty good summary. Thank you for working on this. Just a few things:

The rm commands are only for a complete reset/retry. You also will need to delete Hue from Ambari. Be careful with this. This should make the node ready to try install again. In my vagrant, I can install without any errors. All services are running, I do not have to touch any files. This repo was the bare minimal changes required to install.

[steven-matison]

What errors did you get with this attempt?

[kundeng]

In my last attempt, I saw this: Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/3.1/services/HUE/package/scripts/hue_server.py", line 76, in HueServer().execute() File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute method(env) File "/var/lib/ambari-agent/cache/stacks/HDP/3.1/services/HUE/package/scripts/hue_server.py", line 14, in install import params File "/var/lib/ambari-agent/cache/stacks/HDP/3.1/services/HUE/package/scripts/params.py", line 321, in hbase_master_hosts = default(config['clusterHostInfo']['hbase_master_hosts'],[]) File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/default.py", line 29, in default subdicts = filter(None, name.split('/')) AttributeError: 'list' object has no attribute 'split'

[steven-matison]

Thats weird. If hbase is working and running it should install without errors. To get around that error you literally just go to:

/var/lib/ambari-agent/cache/stacks/HDP/3.1/services/HUE/package/scripts/params.py (line 321) during the installation and start commenting lines and/or modify variables manually until the RETRY succeeds (retry not reinstall).

I think what you want is to comment old line add new line like this:

#hbase_master_hosts = default(config['clusterHostInfo']['hbase_master_hosts'],[])
hbase_master_hosts = ['localhost']

You can also just comment the entire hbase section like we did for spark in the other thread. The install is trying to use the ambari config object to prefill the psuedo distributed hue ini (from HDP2) during the ambari installation steps. Since we are just trying to install (not setup the plugins), that config mapping isn't actually required for hue to run. Later once install is completed, you go into ambari and edit the hue.ini template with the different plugin values you want...

[steven-matison]

@kundeng I just finishing up a hue install using the dfhz_ddp_mpack (same source as the 4.6.0 branch of the HDP3 Hue Service). When I did my install I had no yarn, spark, hbase, etc. To satisfy install errors I just commented the entire yarn and spark section. Next I had a dependency install error for yum install npm. I added epel repository, and hue installed completely. When I get a chance I am going to fix the params file so it doesn't cause any errors. I appreciate your feedback.

[steven-matison]

@kundeng Regarding: "notes 2. the installation took a long time and it also used a lot of memory."

The install is a full make install from GitHub source:

Execute['cd /usr/local/hue-4.6.0 && PREFIX=/usr/local make install'] {}

Depending on your server, the command can take 15-30 minutes. I have some ideas to make this faster, but so far has not been a priority.