search

Nithinraot / SEBC

0 stars 0 forks source link

Resource labs #3

Open Nithinraot opened 6 years ago

Nithinraot commented 6 years ago

Working on Yarn related issues/configurations, a. Yarn/RM lab -> doing the math b. Yarn/RM lab -> reviewing the tuning guides c. Yarn/RM lab -> static service pools d. Yarn/RM lab -> tuning for yarn

Nithinraot commented 6 years ago

YARN/RM Lab: Static Service Pools ->

  1. Navigated to the Static Service Pools page under "cluster"
  2. On the Configuration tab, allocate 20% to HDFS and 80% to YARN
  3. Completed the wizard and restarted the cluster
Nithinraot commented 6 years ago

YARN/RM Lab: Review the Tuning Guides -> used the excel with the given input to arrive at the output Suppose you have a cluster with 20 worker nodes, each of which has:

28 vcores 128 GiB RAM 12 independent disks available to the DataNode Do the following:

Plug the hardware numbers into spreadsheet Change the percentage of reserved memory for the OS to 10% Set Impala's CPU/memory demand to the minimum recommended value Assume HBase and Solr will not be deployed Determine an appropriate workload factor for an ingestion-heavy use case Capture your finished worksheet as a screenshot to resources/labs/1_YarnCalcs.png