search

DanielRamirezSanchez / SEBC

Cloudera Services Enablement Boot Camp - Madrid, October 2018
0 stars 0 forks source link

Installation Lab #1

Open DanielRamirezSanchez opened 5 years ago

DanielRamirezSanchez commented 5 years ago

To ensure OS compatibility, I'm going to install Cloudera Manager 5.13, since my machines are running CentOs 7.4, wich is not compatible with CM 5.8

DanielRamirezSanchez commented 5 years ago

Installed MySQL with databases and user permissions defined in the documentation (https://www.cloudera.com/documentation/enterprise/5-13-x/topics/cm_ig_mysql.html)

DanielRamirezSanchez commented 5 years ago

The lab states that for CentOS 7.x we have to use MariaDB instead of MySQL, but when I try to install MariaDB it says that it's deprecated by mysql (probably because I added previously mysql to the repo when I did a previous installation of mysql, originally I guess it would install mariadb). I'm staying with MySQL since it is already installed.

DanielRamirezSanchez commented 5 years ago

Installed MySQL in al machines, and MySQL server in two machines, the one that's going to be the master, and the replica one. Both have been secured with /usr/bin/mysql_secure_installation. JDBC connector downloaded in all nodes.

DanielRamirezSanchez commented 5 years ago

Replica working, the Slave_IO_State field reads 'Waiting for master to send event'. Copied the result into labs/2_replica_working.md

DanielRamirezSanchez commented 5 years ago

Information required copied to the labs/1_preinstall.md document

DanielRamirezSanchez commented 5 years ago

I had several problems because the cloudera-server was not getting the heartbeats from the cloudera-agents. I did a lot of things to try to solve this:

At the end, after all this still wasn't working. I uninstalled Cloudera, installed it again and when selecting the hosts, I used the FQDN and everything whent smoothly after that

DanielRamirezSanchez commented 5 years ago

Some minor problems with the hostnames (solved with hostnamectl set-hostname "" --static).

Also the files where underreplicated, since only 2 hosts are being used as DataNodes. Moved the replication factor to 2 and solved the issue: sudo -u hdfs hdfs dfs -setrep -R -w 2 / sudo -u hdfs hdfs fsck / -delete

After that, the cluster became healthy. Took the picture of the home page.

DanielRamirezSanchez commented 5 years ago

This morning I realised that the version installed was Cloudera Manager 5.15, not 5.13... Reinstalling everything again.

(Modified cloudera-manager.repo to go get the 5.13 versions, instead of 5.15 by default)

rsiwicki commented 5 years ago

Nice :-)

Note that on production clusters you'd want ensure that the noatime is configured on the data directories of the data nodes.