search

ing-bank / apache-ranger-s3-plugin

Apache Ranger Plugin for S3
Apache License 2.0
19 stars 13 forks source link
apache-ranger aws-s3 ranger-plugin rokku

readme

Build Status codecov.io

Apache Ranger S3 Plugin

Ranger S3 Plugin enables creation of policies in Apache Ranger for S3 buckets hosted on Ceph/RadosGW (S3 coming later). It merely allows for the creation of policies and does not set ACLs by itself. It can be used together with its sister scala project Airlock.

Installation

  1. Run ceph demo container using docker-compose

    docker-compose up

    It will start Ceph demo image on port 8010.

  2. Build the plugin jar using maven 

    mvn package

    If you do not have a local Ceph installation to test against use mvn package -DskipTests.

  3. Copy the jar to ${RANGER_HOME}/ews/webapp/WEB-INF/classes/ranger-plugins/s3. Please note that the location is important (s3).

  4. Load the service definition into Apache Ranger. 

    curl -u <admin>:<admin> -d "@s3-ranger.json" -X POST -H "Accept: application/json" -H "Content-Type: application/json" 
http://{RANGER_HOST}:{RANGER_PORT}/service/public/v2/api/servicedef

  5. Configure the service in Apache Ranger by logging in to the Web UI.

Roadmap

Ranger admin site configuration

Ranger S3 plugin uses aws sdk to connect to backend to list buckets and folders. In case of timeouts, check ranger setting and adjust accordingly. Default 1000 may be to low in some cases. Configuration file is in ranger-1.1.0-admin/ews/webapp/WEB-INF/classes/conf/ranger-admin-default-site.xml.

<property>
    <name>ranger.resource.lookup.timeout.value.in.ms</name>
    <value>10000</value>
    <description />
</property>

Additionally plugin can be configured with different aws region. In order to change region, add following section to ranger-1.1.0-admin/ews/webapp/WEB-INF/classes/conf/ranger-admin-default-site.xml.

 <property>
     <name>airlock.s3.aws.region</name>
     <value>region_name</value>
 </property>

Plugin usage with Airlock

If you run plugin via Airlock, make sure that: