search

Isilon / isilon_hadoop_tools

Tools for Using Hadoop with OneFS
https://pypi.org/project/isilon-hadoop-tools/
MIT License
14 stars 21 forks source link

Fix several issues in user and directory scripts #38

Closed ro-ket closed 7 years ago

ro-ket commented 7 years ago

Run to confirm previous errors are gone:

roket-naver-1# ~/isilon_create_users.sh --dist cdh --zone System
Info: Hadoop distribution:  cdh
Info: will put users in zone:  System
Info: HDFS root:  /ifs/hdfs-root
Info: passwd file: System.passwd
Info: group file: System.group
SUCCESS -- Hadoop users created successfully!
Done!
roket-naver-1# ~/isilon_create_directories.sh --dist cdh --zone System
Info: Hadoop distribution:  cdh
Info: will use users in zone:  System
Info: HDFS root dir is /ifs/hdfs-root
##################################################################################
## Creates Hadoop directory structure on Isilon system HDFS.
##################################################################################
DEBUG: specs dirname /hbase; perm 755; owner hbase; group hbase
DEBUG: specs dirname /solr; perm 775; owner solr; group solr
DEBUG: specs dirname /tmp; perm 1777; owner hdfs; group supergroup
DEBUG: specs dirname /tmp/logs; perm 1777; owner mapred; group hadoop
DEBUG: specs dirname /tmp/hive; perm 777; owner hive; group supergroup
DEBUG: specs dirname /user; perm 755; owner hdfs; group supergroup
DEBUG: specs dirname /user/history; perm 777; owner mapred; group hadoop
DEBUG: specs dirname /user/hive; perm 775; owner hive; group hive
DEBUG: specs dirname /user/hive/warehouse; perm 1777; owner hive; group hive
DEBUG: specs dirname /user/hue; perm 755; owner hue; group hue
DEBUG: specs dirname /user/hue/.cloudera_manager_hive_metastore_canary; perm 777; owner hue; group hue
DEBUG: specs dirname /user/impala; perm 775; owner impala; group impala
DEBUG: specs dirname /user/oozie; perm 775; owner oozie; group oozie
DEBUG: specs dirname /user/flume; perm 775; owner flume; group flume
DEBUG: specs dirname /user/spark; perm 751; owner spark; group spark
DEBUG: specs dirname /user/spark/applicationHistory; perm 1777; owner spark; group spark
DEBUG: specs dirname /user/sqoop2; perm 775; owner sqoop2; group sqoop
SUCCESS -- Hadoop admin directory structure exists and has correct ownership and permissions
Done!
roket-naver-1# ls -al /ifs/hdfs-root
total 14
drwxr-xr-x     6 hdfs   hadoop       88 Aug  3 16:52 .
drwxrwxrwx     6 root   wheel       100 Aug  3 07:34 ..
drwxr-xr-x     2 hbase  hbase         0 Aug  3 16:52 hbase
drwxrwxr-x     2 solr   solr          0 Aug  3 16:52 solr
drwxrwxrwt     4 hdfs   supergroup   44 Aug  3 16:52 tmp
drwxr-xr-x    10 hdfs   supergroup  185 Aug  3 16:52 user
ro-ket commented 7 years ago

Confirmed admin user creation in non-System zone.

roket-naver-1# ~/isilon_create_users.sh --dist cdh --zone azone
Info: Hadoop distribution:  cdh
Info: will put users in zone:  azone
Info: HDFS root:  /ifs/azone
Info: passwd file: azone.passwd
Info: group file: azone.group
SUCCESS -- Hadoop users created successfully!
Done!
roket-naver-1# isi auth users list --zone azone
Name
------------
Guest
hdfs
mapred
yarn
HTTP
hbase
hive
impala
hue
cloudera-scm
accumulo
flume
httpfs
apache
kafka
kms
keytrustee
kudu
llama
oozie
solr
spark
sentry
sqoop
sqoop2
zookeeper
anonymous
cmjobuser
admin
root
nobody
------------
Total: 31