search

rovelynbancolo / SEBC

0 stars 0 forks source link

Security #3

Open rovelynbancolo opened 6 years ago

rovelynbancolo commented 6 years ago
  1. Install Kerberos

yum -y install rng-tools ( RANDOM NUMBER GENERATOR )

systemctl enable rng-tools

yum -y install krb5-server krb5-libs pam_krb5

cd /var/lib/kerberos/krb5kdc

modify the kadm5.acl /admin@CLOUDERA.COM modify the kdc.conf [kdcdefaults] kdc_ports = 88 kdc_tcp_ports = 88

[realms]
 CLOUDERA.COM = {
  #master_key_type = aes256-cts
  acl_file = /var/kerberos/krb5kdc/kadm5.acl
  dict_file = /usr/share/dict/words
  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
  supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-$
}

***modify the /etc/krb5.conf
[libdefaults] default_realm = CLOUDERA.COM dns_lookup_kdc = false dns_lookup_realm = false ticket_lifetime = 604800 renew_lifetime = 604800 forwardable = true default_tgs_enctypes = rc4-hmac default_tkt_enctypes = rc4-hmac permitted_enctypes = rc4-hmac udp_preference_limit = 1 kdc_timeout = 3000 [realms] CLOUDERA.COM = { kdc = ip-172-31-0-136.ap-southeast-1.compute.internal admin_server = ip-172-31-0-136.ap-southeast-1.compute.internal } [domain_realm]

****Creating an initial KDC database

#krb5_util create -s -r CLOUDERA.COM

systemctl enable krb5kdc kadmin

systemctl start krb5kdc kadmin

chkconfig krb5kdc on

chkconfig kadmin on

kadmin.local "addprinc -q admin/admin"

Enter password for this admin principal

Kerberos Ticket Lifetime ticket_lifetime : 7days

KDC Type :MIT KDC

Kerberos Security Realm default_realm CLOUDERA.COM

KDC Server Host kdc ip-172-31-0-136.ap-southeast-1.compute.internal

KDC Admin Server Host admin_server ip-172-31-0-136.ap-southeast-1.compute.internal

Manage krb5.conf through Cloudera Manager

HUE: Kerberos Ticket Renewer Health Suppress... Healthy Kerberos Ticket Renewer: 0. Concerning Kerberos Ticket Renewer: 0. Total Kerberos Ticket Renewer: 1. Percent healthy: 0.00%. Percent healthy or concerning: 0.00%. Critical threshold: 51.00%.

kadmin -p admin/admin

listprincs

modprinc -maxrenewlife 90days krbtgt/CLOUDERA.COM@CLOUDERA.COM modprinc -maxrenewlife 90days +allow_renewable hue/ip-172-31-3-221.ap-southeast-1.compute.internal@CLOUDERA.COM

CM -> HUE -> Instances -> Stop HUE Server -> Click on it -> Regenerate Keytab. Restart the services.

HUE YARN - Enable Kerberos Authentication for HTTP Web-Consoles HDFS Enable Kerberos Authentication for HTTP Web-Consoles

Sentry:

On all host added this user and group details

sudo groupadd selector

sudo groupadd inserters

sudo useradd -u 1100 -g selector george

sudo useradd -u 1200 -g inserters ferdinand

useradd rovy

groupadd data_engineers

usermod -g data_engineers rovy

kadmin -p admin/admin

addprinc george@CLOUDERA.COM

addprinc ferdinand@CLOUDERA.COM

addprinc rovy@CLOUDERA.COM

addprinc root@CLOUDERA.COM

addprinc hdfs@CLOUDERA.COM

[rovy@ip-172-31-0-136 ~]kinit rovy@CLOUDERA.COM [rovy@ip-172-31-0-136 ~] hdfs dfs -copyFromLocal *.csv /user/rovy/

Create a Hive database using HUE -> named the table insurance_profile. Access beeline.

beeline

!connect 'jdbc:hive2://ip-172-31-0-136.ap-southeast-1.compute.internal:10000/;principal=hive/ip-172-31-0-136.ap-southeast-1.compute.internal@CLOUDERA.COM'

load data inpath '/user/hive/warehouse/datalake.db/insurance_portfolio.csv' into table datalake.insurance_portfolio;

Enable Sentry

  1. Create a Sentry Database

    CREATE DATABASE sentry default character set utf8;

    GRANT ALL ON sentry.* to 'sentry'@'%' identified by 'sentry_password';

    flush privileges;

  2. CM -> Add SEntry -> Enter DB Details

    **Sentry Server Error -> it doesnt create table to the database*** Tried Creating the Database, remove sentry and readd it again. Query for candidates of org.apache.sentry.provider.db.service.model.MSentryVersion and subclasses resulted in no possible candidates Required table missing : "SENTRY_VERSION" in Catalog "" Schema "". DataNucleus requires this table to perform its persistence operations. Either your MetaData is incorrect, or you need to enable "datanucleus.autoCreateTables" org.datanucleus.store.rdbms.exceptions.MissingTableException: Required table missing : "SENTRY_VERSION" in Catalog "" Schema "". DataNucleus requires this table to perform its persistence operations. Either your MetaData is incorrect, or you need to enable "datanucleus.autoCreateTables"

--> Still Unable to Get Sentry Working.

rovelynbancolo commented 6 years ago

Sentry:

Configure Sentry to recognize this account as an administrator Add your test user's primary group to the sentry.service.admin.group list in CM

rovy: data_engineers Added data_engineers to sentry.service.admin.group

rovelynbancolo commented 6 years ago

Verify user privileges

beeline

!connect 'jdbc:hive2://ip-172-31-0-136.ap-southeast-1.compute.internal:10000/;principal=hive/ip-

172-31-0-136.ap-southeast-1.compute.internal@CLOUDERA.COM'

0: jdbc:hive2://ip-172-31-0-136.ap-southeast-> show tables;

 +-----------+--+

| tab_name | +-----------+--+ +-----------+--+ No rows selected (0.36 seconds) 0: jdbc:hive2://ip-172-31-0-136.ap-southeast->

rovelynbancolo commented 6 years ago

Create additional test users

On all host added this user and group details

sudo groupadd selector

sudo groupadd inserters

sudo useradd -u 1100 -g selector george

sudo useradd -u 1200 -g inserters ferdinand

useradd rovy

groupadd data_engineers

usermod -g data_engineers rovy

kadmin -p admin/admin

addprinc george@CLOUDERA.COM addprinc ferdinand@CLOUDERA.COM addprinc rovy@CLOUDERA.COM addprinc root@CLOUDERA.COM addprinc hdfs@CLOUDERA.COM

rovelynbancolo commented 6 years ago

Create test roles

beeline

!connect 'jdbc:hive2://ip-172-31-0-136.ap-southeast-1.compute.internal:10000/;principal=hive/ip-172-31-0-136.ap-southeast-1.compute.internal@CLOUDERA.COM' CREATE ROLE reads; CREATE ROLE writes;

rovelynbancolo commented 6 years ago

Grant read privilege for all tables to reads

GRANT SELECT ON DATABASE default TO ROLE reads; GRANT ROLE reads TO GROUP selector;

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.SentryGrantRevokeTask. SentryAccessDeniedException: root has no grant! (state=08S01,code=1)

su rovy

kinit rovy@CLOUDERA.COM

beeline

!connect 'jdbc:hive2://ip-172-31-0-136.ap-southeast-1.compute.internal:10000/;principal=hive/ip-172-31-0-136.ap-southeast-1.compute.internal@CLOUDERA.COM'

GRANT SELECT ON DATABASE default TO ROLE reads; GRANT ROLE reads TO GROUP selector;

rovelynbancolo commented 6 years ago

Grant read privilege for default.sample07 only to 'writes': Create a table under default. CREATE EXTERNAL TABLE IF NOT EXISTS default.sample_07( policyid VARCHAR(255) ,statecode VARCHAR(255) ,county VARCHAR(255) ,eq_site_limit VARCHAR(255) ,hu_site_limit VARCHAR(255) ,fl_site_limit VARCHAR(255) ,tiv_2011 VARCHAR(255) ,tiv_2012 VARCHAR(255) ,eq_site_deductible VARCHAR(255) ,hu_site_deductible VARCHAR(255) ,fl_site_deductible VARCHAR(255) ,fr_site_deductible VARCHAR(255) ,point_latitude VARCHAR(255) ,point_longitude VARCHAR(255) ,line VARCHAR(255) ,construction VARCHAR(255) ,point_granularity VARCHAR(255)) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE location '/user/hive/insurance_portfolio.csv';

load data inpath 'hdfs://ip-172-31-0-136.ap-southeast-1.compute.internal:8020/rovy/insurance_portfolio.csv' into table default.sample_07

kinit rovy@CLOUDERA.COM

!connect 'jdbc:hive2://ip-172-31-0-136.ap-southeast-1.compute.internal:10000/;principal=hive/ip-172-31-0-136.ap-southeast-1.compute.internal@CLOUDERA.COM'

REVOKE ALL ON DATABASE default FROM ROLE writes; GRANT SELECT ON default.sample_07 TO ROLE writes; GRANT ROLE writes TO GROUP inserters;

rovelynbancolo commented 6 years ago

kinit as george, then login to beeline [rovy@ip-172-31-0-136 logs]$ kinit george@CLOUDERA.COM Password for george@CLOUDERA.COM: [rovy@ip-172-31-0-136 logs]$ klist Ticket cache: FILE:/tmp/krb5cc_1002 Default principal: george@CLOUDERA.COM

Valid starting Expires Service principal 05/17/2018 04:34:45 05/18/2018 04:34:45 krbtgt/CLOUDERA.COM@CLOUDERA.COM

beeline

!connect 'jdbc:hive2://ip-172-31-0-136.ap-southeast-1.compute.internal:10000/default;principal=hive/ip-172-31-0-136.ap-southeast-1.compute.internal@CLOUDERA.COM'

0: jdbc:hive2://ip-172-31-0-136.ap-southeast-> show tables; INFO : Compiling command(queryId=hive_20180517043939_fa110382-da27-474f-ac90-95960923d267): show tables INFO : Semantic Analysis Completed INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null) INFO : Completed compiling command(queryId=hive_20180517043939_fa110382-da27-474f-ac90-95960923d267); Time taken: 0.049 seconds INFO : Executing command(queryId=hive_20180517043939_fa110382-da27-474f-ac90-95960923d267): show tables INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20180517043939_fa110382-da27-474f-ac90-95960923d267); Time taken: 0.115 seconds INFO : OK +------------+--+ | tab_name | +------------+--+ | sample_07 | | sample_08 | +------------+--+ 2 rows selected (0.207 seconds) 0: jdbc:hive2://ip-172-31-0-136.ap-southeast->

[rovy@ip-172-31-0-136 logs]$ kinit ferdinand@CLOUDERA.COM Password for ferdinand@CLOUDERA.COM: [rovy@ip-172-31-0-136 logs]$ klist Ticket cache: FILE:/tmp/krb5cc_1002 Default principal: ferdinand@CLOUDERA.COM

Valid starting Expires Service principal 05/17/2018 04:41:17 05/18/2018 04:41:17 krbtgt/CLOUDERA.COM@CLOUDERA.COM [rovy@ip-172-31-0-136 logs]$ beeline

show tables;

+------------+--+ | tab_name | +------------+--+ | sample_07 | +------------+--+