bluishglc / ranger-emr-cli-installer

This is a powerful cli tool for Apache Ranger and AWS EMR automated installation & integration with OpenLDAP & Windows AD. It supports Open-Source Ranger and EMR-Native Ranger both, supports OpenLDAP & Windows AD both, and works in all AWS regions (also including China regions).
8 stars 15 forks source link

Solr Permission denied with OpenLDAP + Open-Source Ranger example #2

Closed heng2j closed 1 year ago

heng2j commented 1 year ago

Hi Laurence,

Thank you so much for putting together this wonderful repo and the tutorial on DZone. I am following the "Apache Ranger and AWS EMR Automated Installation and Integration Series (4): OpenLDAP + Open-Source Ranger", and I got stuck on step 2.3.3 Install Ranger with permission denied on starting Solr.

Here the log output of what I am seeing now.

======================================================================================    INIT SOLR AS RANGER AUDIT STORE    ======================================================================================

Fri Mar  3 00:13:18 CST 2023|INFO|Solr Ranger Home </opt/solr/ranger_audit_server> exists. Will overwrite configurations
Fri Mar  3 00:13:18 CST 2023|INFO|Configuring standalone instance
Fri Mar  3 00:13:18 CST 2023|INFO|Copying Ranger Audit Server configuration to /opt/solr/ranger_audit_server
Fri Mar  3 00:13:18 CST 2023|INFO|Done configuring Solr for Apache Ranger Audit
Fri Mar  3 00:13:18 CST 2023|INFO|Solr HOME for Ranger Audit is /opt/solr/ranger_audit_server
Fri Mar  3 00:13:18 CST 2023|INFO|Data folder for Audit logs is /opt/solr/ranger_audit_server/data
Fri Mar  3 00:13:18 CST 2023|INFO|To start Solr run /opt/solr/ranger_audit_server/scripts/start_solr.sh
Fri Mar  3 00:13:18 CST 2023|INFO|To stop Solr run /opt/solr/ranger_audit_server/scripts/stop_solr.sh
Fri Mar  3 00:13:18 CST 2023|INFO|After starting Solr for RangerAudit, it will listen at 8983. E.g http://ip-____.ec2.internal:8983
Fri Mar  3 00:13:18 CST 2023|INFO|Configure Ranger to use the following URL http://ip-____.ec2.internal:8983/solr/ranger_audits
Fri Mar  3 00:13:18 CST 2023|INFO| ** NOTE: If Solr is Secured then solrclient JAAS configuration has to be added to Ranger Admin and Ranger Plugin properties
Fri Mar  3 00:13:18 CST 2023|INFO| ** Refer documentation on how to configure Ranger for audit to Secure Solr
########## Done ###################
Created file /opt/solr/ranger_audit_server/install_notes.txt with instructions to start and stop
###################################
No process found for Solr node running on port 8983
/opt/solr/bin/solr: line 2246: /opt/solr/bin/solr-8983.pid: Permission denied
Warning: Available entropy is low. As a result, use of the UUIDField, SSL, or any other features that require
RNG might not work properly. To check for the amount of available entropy, use 'cat /proc/sys/kernel/random/entropy_avail'.

Waiting up to 180 seconds to see Solr running on port 8983 [\]  Still not seeing Solr listening on 8983 after 180 seconds!
tail: cannot open ‘/var/log/solr/ranger_audits/solr.log’ for reading: No such file or directory

============================================================================    TEST SOLR CONNECTIVITY    ============================================================================

I was running the following shell command on my Ranger server EC2 instance.

sudo sh ./ranger-emr-cli-installer/bin/setup.sh install-ranger \
    --region "$REGION" \
    --access-key-id "$ACCESS_KEY_ID" \
    --secret-access-key "$SECRET_ACCESS_KEY" \
    --solution 'open-source' \
    --auth-provider 'openldap' \
    --openldap-host "$OPENLDAP_HOST" \
    --openldap-base-dn 'dc=example,dc=com' \
    --ranger-bind-dn 'cn=ranger,ou=services,dc=example,dc=com' \
    --ranger-bind-password 'Admin1234!' \
    --openldap-user-dn-pattern 'uid={0},ou=users,dc=example,dc=com' \
    --openldap-group-search-filter '(member=uid={0},ou=users,dc=example,dc=com)' \
    --openldap-user-object-class 'inetOrgPerson'

My shell script and linux permissions are rusty, so any insights or recommendation on how to solve this problem will be great.

Thank you, Heng

heng2j commented 1 year ago

The issue may not due to permissions, since I just changed the permissions for the entire /opt/solr/ folder.

By looking at the /var/log/solr/ranger_audits/solr-8983-console.log the issue may due to lack of memory in my t2.micro instance.

OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000a0000000, 1610612736, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 1610612736 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /opt/solr-8.6.2/server/hs_err_pid20672.log
bluishglc commented 1 year ago

hi @heng2j , it seems your host has no enough memory to start solr. it is recommended that 4 vCore / 16 GB or above.

heng2j commented 1 year ago

Yes indeed @bluishglc! Thank you for your confirmation.