Challenge Setup

[Nithinraot]

Create the Issue Challenges Setup Make sure manojsundaram and melvin-koh are Collaborators Assign the Issue to yourself and label it started Destroy your lab instances and create 5 new instances (Follow the same instructions from the lab, plus make sure your root volume EBS storage has 50 GB allocated). In the file challenges/labs/0_setup.md: List the cloud provider you are using List your instances by IP address and DNS name (don't use /etc/hosts for this) List the Linux release you are using List the file system capacity for the first node List the command and output for yum repolist enabled Add the following Linux accounts to all nodes User jimenez with a UID of 2800 User beltran with a UID of 2900 Create the group astros and add beltran to it Create the group rangers and add jimenez to it List the /etc/passwd entries for jimenez and beltran Do not list the entire file List the /etc/group entries for astros and rangers Do not list the entire file

[Nithinraot]

1.Created the instances (5).

2. disabled selinux & ipv6 in all 5 isntances -> /etc/
3. [root@ip-172-31-38-16 ec2-user]# echo '1' > /proc/sys/vm/swappiness
4. hugepage compaction disabling echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag, echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled
5. verified forward and reverse sync nslookup ip-172-31-38-16 Server: 172.31.0.2 Address: 172.31.0.2#53

[Nithinraot]

6. verifiying ifconfig -> [root@ip-172-31-38-16 ec2-user]# ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9001 inet 172.31.38.16 netmask 255.255.240.0 broadcast 172.31.47.255 inet6 fe80::828:b6ff:fefa:580e prefixlen 64 scopeid 0x20 ether 0a:28:b6:fa:58:0e txqueuelen 1000 (Ethernet) RX packets 42649 bytes 62697467 (59.7 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 7596 bytes 684392 (668.3 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10 loop txqueuelen 1000 (Local Loopback) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

7. mount -> /dev/xvda2 on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)

[Nithinraot]

install Java -> wget "http://archive.cloudera.com/director/redhat/7/x86_64/director/cloudera-director.repo" [root@ip-172-31-5-153 ec2-user]# yum list all | grep oracle-* oracle-j2sdk1.8.x86_64 1.8.0+update121-1

yum install oracle-j2sdk1.8.x86_64

export JAVA_HOME=/usr/java/jdk1.8.0_121-cloudera export JRE_HOME=/usr/java/jdk1.8.0_121-cloudera/jre export PATH=/usr/java/jdk1.8.0_121-cloudera/bin/:/usr/java/jdk1.8.0_121-cloudera/jre/bin/

[Nithinraot]

Added the users & groups -> [root@ip-172-31-38-16 ec2-user]# useradd -u 2800 jimenez [root@ip-172-31-38-16 ec2-user]# useradd -u 2900 beltran [root@ip-172-31-38-16 ec2-user]# groupadd astros [root@ip-172-31-38-16 ec2-user]# groupadd rangers [root@ip-172-31-38-16 ec2-user]# usermod -G astros beltran [root@ip-172-31-38-16 ec2-user]# usermod -G rangers jimenez [root@ip-172-31-38-16 ec2-user]# id jimenez uid=2800(jimenez) gid=2800(jimenez) groups=2800(jimenez),2902(rangers) [root@ip-172-31-38-16 ec2-user]# id beltran uid=2900(beltran) gid=2900(beltran) groups=2900(beltran),2901(astros) [root@ip-172-31-38-16 ec2-user]# tail -2 /etc/passwd jimenez:x:2800:2800::/home/jimenez:/bin/bash beltran:x:2900:2900::/home/beltran:/bin/bash [root@ip-172-31-38-16 ec2-user]# tail -2 /etc/group astros:x:2901:beltran rangers:x:2902:jimenez

[Nithinraot]

installing maria db: sudo yum install mariadb-server sudo service mariadb stop update /etc/my.cnf -> socket=/var/lib/mysql/mysql.sock

Disabling symbolic-links is recommended to prevent assorted security risks

symbolic-links=0

Settings user and group are ignored when systemd is used. If you need to run mysqld under a different user or group, customize your systemd unit file for mariadb according to the instructions in http://fedoraproject.org/wiki/Systemd key_buffer = 16M key_buffer_size = 32M max_allowed_packet = 32M thread_stack = 256K thread_cache_size = 64 query_cache_limit = 8M query_cache_size = 64M query_cache_type = 1

binlog_format = mixed

read_buffer_size = 2M read_rnd_buffer_size = 16M sort_buffer_size = 8M join_buffer_size = 8M

log_bin=/var/lib/mysql/mysql_binary_log max_connections = 550 innodb_file_per_table = 1 innodb_flush_log_at_trx_commit = 2 innodb_log_buffer_size = 64M innodb_buffer_pool_size = 4G innodb_thread_concurrency = 8 innodb_flush_method = O_DIRECT innodb_log_file_size = 512M

[mysqld_safe] log-error=/var/log/mariadb/mariadb.log pid-file=/var/run/mariadb/mariadb.pid

include all files from the config directory !includedir /etc/my.cnf.d

sudo systemctl enable mariadb sudo service mariadb start sudo /usr/bin/mysql_secure_installation -> here set the username password for root/ create a DB named SCM & user named SCM where cloudera-manager-server will use this db.

get the mysql-connector -> wget "https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz"

tar -xvf mysql-connector-java-5.1.46.tar.gz

copy this connector to /user/share/java ( create this folder if it doesn not exists) cp mysql-connector-java-5.1.46 /user/share/java/mysql-connector-java

[Nithinraot]

Installing cloudera manager

wget https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/cloudera-manager.repo yum list all | grep cloudera-manager yum install cloudera-manager-server Test the DB connection & also check if the required tables are created - > sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql -h ip-172-31-5-153 scm scm If the above test succeeds then run systemctl start cloudera-scm-server

[Nithinraot]

[root@ip-172-31-38-16 schema]# ./scm_prepare_database.sh mysql -h ip-172-31-38-16 scm scm Enter SCM password: JAVA_HOME=/usr/java/jdk1.8.0_121-cloudera Verifying that we can write to /etc/cloudera-scm-server Creating SCM configuration file in /etc/cloudera-scm-server Executing: /usr/java/jdk1.8.0_121-cloudera/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db. [ main] DbCommandExecutor INFO Successfully connected to database. All done, your SCM database is configured correctly!