--append-cluster-name is not always respected in the CDH special cases

[dmtucker]

In the CDH path, sqoop and impala are added to groups (without the cluster name appended):

# use this file to add to the group file of your clients
hdfs-clustername:x:1000:
mapred-clustername:x:1001:
yarn-clustername:x:1002:
HTTP-clustername:x:1003:
cloudera-scm-clustername:x:1004:
accumulo-clustername:x:1005:
flume-clustername:x:1006:
hbase-clustername:x:1007:
httpfs-clustername:x:1009:
hue-clustername:x:1010:
apache-clustername:x:1011:
impala-clustername:x:1012:
kafka-clustername:x:1013:
kms-clustername:x:1014:
keytrustee-clustername:x:1015:
kudu-clustername:x:1016:
llama-clustername:x:1017:
oozie-clustername:x:1018:
solr-clustername:x:1019:
spark-clustername:x:1020:
sentry-clustername:x:1021:
sqoop-clustername:x:1022:
zookeeper-clustername:x:1024:
anonymous-clustername:x:1025:
cmjobuser-clustername:x:1026:
hadoop-clustername:x:1027:,hdfs-clustername,mapred-clustername,yarn-clustername,HTTP-clustername
supergroup-clustername:x:1028:,hdfs-clustername,mapred-clustername,yarn-clustername,HTTP-clustername
sqoop2-clustername:x:1023:,sqoop
hive-clustername:x:1008:,impala

Those users don't actually exist according to the passwd file:

# use this file to add to the passwd file of your clients
hdfs-clustername:x:1000:1000:hadoop-svc-account:/home/hdfs-clustername:/bin/bash
mapred-clustername:x:1001:1001:hadoop-svc-account:/home/mapred-clustername:/bin/bash
yarn-clustername:x:1002:1002:hadoop-svc-account:/home/yarn-clustername:/bin/bash
HTTP-clustername:x:1003:1003:hadoop-svc-account:/home/HTTP-clustername:/bin/bash
cloudera-scm-clustername:x:1004:1004:hadoop-svc-account:/home/cloudera-scm-clustername:/bin/bash
accumulo-clustername:x:1005:1005:hadoop-svc-account:/home/accumulo-clustername:/bin/bash
flume-clustername:x:1006:1006:hadoop-svc-account:/home/flume-clustername:/bin/bash
hbase-clustername:x:1007:1007:hadoop-svc-account:/home/hbase-clustername:/bin/bash
hive-clustername:x:1008:1008:hadoop-svc-account:/home/hive-clustername:/bin/bash
httpfs-clustername:x:1009:1009:hadoop-svc-account:/home/httpfs-clustername:/bin/bash
hue-clustername:x:1010:1010:hadoop-svc-account:/home/hue-clustername:/bin/bash
apache-clustername:x:1011:1011:hadoop-svc-account:/home/apache-clustername:/bin/bash
impala-clustername:x:1012:1012:hadoop-svc-account:/home/impala-clustername:/bin/bash
kafka-clustername:x:1013:1013:hadoop-svc-account:/home/kafka-clustername:/bin/bash
kms-clustername:x:1014:1014:hadoop-svc-account:/home/kms-clustername:/bin/bash
keytrustee-clustername:x:1015:1015:hadoop-svc-account:/home/keytrustee-clustername:/bin/bash
kudu-clustername:x:1016:1016:hadoop-svc-account:/home/kudu-clustername:/bin/bash
llama-clustername:x:1017:1017:hadoop-svc-account:/home/llama-clustername:/bin/bash
oozie-clustername:x:1018:1018:hadoop-svc-account:/home/oozie-clustername:/bin/bash
solr-clustername:x:1019:1019:hadoop-svc-account:/home/solr-clustername:/bin/bash
spark-clustername:x:1020:1020:hadoop-svc-account:/home/spark-clustername:/bin/bash
sentry-clustername:x:1021:1021:hadoop-svc-account:/home/sentry-clustername:/bin/bash
sqoop-clustername:x:1022:1022:hadoop-svc-account:/home/sqoop-clustername:/bin/bash
sqoop2-clustername:x:1023:1023:hadoop-svc-account:/home/sqoop2-clustername:/bin/bash
zookeeper-clustername:x:1024:1024:hadoop-svc-account:/home/zookeeper-clustername:/bin/bash
anonymous-clustername:x:1025:1025:hadoop-svc-account:/home/anonymous-clustername:/bin/bash
cmjobuser-clustername:x:1026:1026:hadoop-svc-account:/home/cmjobuser-clustername:/bin/bash

This seems to be specific to the client files (i.e. we do not try to create these users or add them to groups on OneFS): https://github.com/Isilon/isilon_hadoop_tools/blob/6cbf7f15d4323aa390b86f8a4832b8654684ac24/isilon_create_users.sh#L297

[bonibruno]

Fixed.

hop-isi-c-1# bash ./icu2.sh --dist cdh --zone cdh --append-cluster-name test
Info: Hadoop distribution:  cdh
Info: will put users in zone:  cdh
Info: will add clustername to end of usernames: -test
Info: HDFS root:  /ifs/cdh
Info: passwd file: cdh.passwd
Info: group file: cdh.group
SUCCESS -- Hadoop users created successfully!
Done!

hop-isi-c-1# cat cdh.group
# use this file to add to the group file of your clients
hdfs-test:x:1000:
mapred-test:x:1001:
yarn-test:x:1002:
HTTP-test:x:1003:
hbase-test:x:1004:
impala-test:x:1006:
hue-test:x:1007:
cloudera-scm-test:x:1008:
accumulo-test:x:1009:
flume-test:x:1010:
httpfs-test:x:1011:
apache-test:x:1012:
kafka-test:x:1013:
kms-test:x:1014:
keytrustee-test:x:1015:
kudu-test:x:1016:
llama-test:x:1017:
oozie-test:x:1018:
solr-test:x:1019:
spark-test:x:1020:
sentry-test:x:1021:
sqoop2-test:x:1023:
zookeeper-test:x:1024:
anonymous-test:x:1025:
cmjobuser-test:x:1026:
hadoop-test:x:1027:,hdfs-test,mapred-test,yarn-test,HTTP-test,hbase-test
supergroup-test:x:1028:,hdfs-test,mapred-test,yarn-test,HTTP-test,hbase-test
sqoop-test:x:1022:,sqoop2-test
hive-test:x:1005:,impala-test

hop-isi-c-1# cat cdh.passwd
# use this file to add to the passwd file of your clients hdfs-test:x:1000:1000:hadoop-svc-account:/home/hdfs-test:/bin/bash
mapred-test:x:1001:1001:hadoop-svc-account:/home/mapred-test:/bin/bash
yarn-test:x:1002:1002:hadoop-svc-account:/home/yarn-test:/bin/bash
HTTP-test:x:1003:1003:hadoop-svc-account:/home/HTTP-test:/bin/bash
hbase-test:x:1004:1004:hadoop-svc-account:/home/hbase-test:/bin/bash
hive-test:x:1005:1005:hadoop-svc-account:/home/hive-test:/bin/bash
impala-test:x:1006:1006:hadoop-svc-account:/home/impala-test:/bin/bash
hue-test:x:1007:1007:hadoop-svc-account:/home/hue-test:/bin/bash
cloudera-scm-test:x:1008:1008:hadoop-svc-account:/home/cloudera-scm-test:/bin/bash
accumulo-test:x:1009:1009:hadoop-svc-account:/home/accumulo-test:/bin/bash
flume-test:x:1010:1010:hadoop-svc-account:/home/flume-test:/bin/bash
httpfs-test:x:1011:1011:hadoop-svc-account:/home/httpfs-test:/bin/bash
apache-test:x:1012:1012:hadoop-svc-account:/home/apache-test:/bin/bash
kafka-test:x:1013:1013:hadoop-svc-account:/home/kafka-test:/bin/bash
kms-test:x:1014:1014:hadoop-svc-account:/home/kms-test:/bin/bash
keytrustee-test:x:1015:1015:hadoop-svc-account:/home/keytrustee-test:/bin/bash
kudu-test:x:1016:1016:hadoop-svc-account:/home/kudu-test:/bin/bash
llama-test:x:1017:1017:hadoop-svc-account:/home/llama-test:/bin/bash
oozie-test:x:1018:1018:hadoop-svc-account:/home/oozie-test:/bin/bash
solr-test:x:1019:1019:hadoop-svc-account:/home/solr-test:/bin/bash
spark-test:x:1020:1020:hadoop-svc-account:/home/spark-test:/bin/bash
sentry-test:x:1021:1021:hadoop-svc-account:/home/sentry-test:/bin/bash
sqoop-test:x:1022:1022:hadoop-svc-account:/home/sqoop-test:/bin/bash
sqoop2-test:x:1023:1023:hadoop-svc-account:/home/sqoop2-test:/bin/bash
zookeeper-test:x:1024:1024:hadoop-svc-account:/home/zookeeper-test:/bin/bash
anonymous-test:x:1025:1025:hadoop-svc-account:/home/anonymous-test:/bin/bash
cmjobuser-test:x:1026:1026:hadoop-svc-account:/home/cmjobuser-test:/bin/bash

[bonibruno]

Fix with pull request https://github.com/Isilon/isilon_hadoop_tools/pull/25

[dmtucker]

Pro tip: If you do this the other way around (put the issue being fixed in the PR instead of vice-versa), GitHub will automatically close the issue when the PR merges. More info: https://github.com/blog/1506-closing-issues-via-pull-requests