Isilon / isilon_hadoop_tools

Tools for Using Hadoop with OneFS
https://pypi.org/project/isilon-hadoop-tools/
MIT License
14 stars 21 forks source link

Update isilon_create_users.sh #32

Closed bonibruno closed 7 years ago

bonibruno commented 7 years ago

This update adds titan user for BI distribution (needed for BI v 4.3) and addresses all current open issues.
Note: The trailing "," in the group file is not a problem.

Below is the output for cdh test. hwx test already tested fine.

bash ./isilon_create_users.sh --dist cdh --zone cdh Info: Hadoop distribution: cdh Info: will put users in zone: cdh Info: HDFS root: /ifs/cdh Info: passwd file: cdh.passwd Info: group file: cdh.group SUCCESS -- Hadoop users created successfully! Done!

cat cdh.passwd

use this file to add to the passwd file of your clients

hdfs:x:1000:1000:hadoop-svc-account:/home/hdfs:/bin/bash mapred:x:1001:1001:hadoop-svc-account:/home/mapred:/bin/bash yarn:x:1002:1002:hadoop-svc-account:/home/yarn:/bin/bash HTTP:x:1003:1003:hadoop-svc-account:/home/HTTP:/bin/bash hbase:x:1004:1004:hadoop-svc-account:/home/hbase:/bin/bash hive:x:1005:1005:hadoop-svc-account:/home/hive:/bin/bash impala:x:1006:1006:hadoop-svc-account:/home/impala:/bin/bash hue:x:1007:1007:hadoop-svc-account:/home/hue:/bin/bash cloudera-scm:x:1008:1008:hadoop-svc-account:/home/cloudera-scm:/bin/bash accumulo:x:1009:1009:hadoop-svc-account:/home/accumulo:/bin/bash flume:x:1010:1010:hadoop-svc-account:/home/flume:/bin/bash httpfs:x:1011:1011:hadoop-svc-account:/home/httpfs:/bin/bash apache:x:1012:1012:hadoop-svc-account:/home/apache:/bin/bash kafka:x:1013:1013:hadoop-svc-account:/home/kafka:/bin/bash kms:x:1014:1014:hadoop-svc-account:/home/kms:/bin/bash keytrustee:x:1015:1015:hadoop-svc-account:/home/keytrustee:/bin/bash kudu:x:1016:1016:hadoop-svc-account:/home/kudu:/bin/bash llama:x:1017:1017:hadoop-svc-account:/home/llama:/bin/bash oozie:x:1018:1018:hadoop-svc-account:/home/oozie:/bin/bash solr:x:1019:1019:hadoop-svc-account:/home/solr:/bin/bash spark:x:1020:1020:hadoop-svc-account:/home/spark:/bin/bash sentry:x:1021:1021:hadoop-svc-account:/home/sentry:/bin/bash sqoop:x:1022:1022:hadoop-svc-account:/home/sqoop:/bin/bash sqoop2:x:1023:1023:hadoop-svc-account:/home/sqoop2:/bin/bash zookeeper:x:1024:1024:hadoop-svc-account:/home/zookeeper:/bin/bash anonymous:x:1025:1025:hadoop-svc-account:/home/anonymous:/bin/bash cmjobuser:x:1026:1026:hadoop-svc-account:/home/cmjobuser:/bin/bash

cat cdh.group

use this file to add to the group file of your clients

hdfs:x:1000: mapred:x:1001: yarn:x:1002: HTTP:x:1003: hbase:x:1004: impala:x:1006: hue:x:1007: cloudera-scm:x:1008: accumulo:x:1009: flume:x:1010: httpfs:x:1011: apache:x:1012: kafka:x:1013: kms:x:1014: keytrustee:x:1015: kudu:x:1016: llama:x:1017: oozie:x:1018: solr:x:1019: spark:x:1020: sentry:x:1021: zookeeper:x:1024: anonymous:x:1025: cmjobuser:x:1026: hadoop:x:1027:hdfs,mapred,yarn,HTTP,hbase, supergroup:x:1028:hdfs,mapred,yarn,HTTP,hbase, sqoop:x:1022:sqoop2 sqoop2:x:1023:sqoop hive:x:1005:impala

dmtucker commented 7 years ago

You probably will want to Squash and Merge this when you are ready ;) Thanks!

bonibruno commented 7 years ago

This update adds titan to BI path, fixes sqoop and sqoop2 group assignments for CDH path, and cleans up created password and group reference files.