AWS Maven Wagon

This project is a fork from https://github.com/spring-projects/aws-maven to support development and operations at Finnish Brodcasting Company ( YLE ). No guarantees are made for support or updating the component, but as long as we are using it actively we will update it as we need it.

Wagon also works with Leiningen out of the box.

Add following to your project.clj

:plugins [[fi.yle.tools/aws-maven "1.5.0"]]
:plugin-repositories [["yle-repo" {:url       "https://maven.yle.fi/release"
                                   :snapshots false}]]

Latest version is 1.5.0 and it uses AWS SDK version 1.12.301

Building and deploying this wagon

mvn install

mvn deploy

Deploy works with Java 8, but doesn't necessarily work with more recent versions without modifications.

Usage

To publish Maven artifacts to S3 a build extension must be defined in a project's pom.xml. The latest version of the wagon can be found from from YLE public mvn repository https://maven.yle.fi/release

To get the dependency add to your pom:

<pluginRepositories>
    <pluginRepository>
      <id>yle-public</id>
      <name>Yle public repository</name>
      <url>https://maven.yle.fi/release</url>
      <layout>default</layout>
    </pluginRepository>
</pluginRepositories>

And plugin dependency:

<project>
  ...
  <build>
    ...
    <extensions>
      ...
      <extension>
      <groupId>fi.yle.tools</groupId>
      <artifactId>aws-maven</artifactId>
      <version>1.4.1</version>
      </extension>
      ...
    </extensions>
    ...
  </build>
  ...
</project>

This allows then using dependencies from s3 repositories as well as publish to s3 repositories.

Once the build extension is configured distribution management repositories can be defined in the pom.xml with an s3:// scheme.

<project>
  ...
  <distributionManagement>
    <repository>
      <id>aws-release</id>
      <name>AWS Release Repository</name>
      <url>s3://<BUCKET>/release</url>
    </repository>
    <snapshotRepository>
      <id>aws-snapshot</id>
      <name>AWS Snapshot Repository</name>
      <url>s3://<BUCKET>/snapshot</url>
    </snapshotRepository>
  </distributionManagement>
  ...
</project>

Finally the ~/.m2/settings.xml should be updated to include access and secret keys for the account. The access key should be used to populate the username element, and the secret access key should be used to populate the password element.

<settings>
  ...
  <servers>
    ...
    <server>
      <id>aws-release</id>
      <username>0123456789ABCDEFGHIJ</username>
      <password>0123456789abcdefghijklmnopqrstuvwxyzABCD</password>
    </server>
    <server>
      <id>aws-snapshot</id>
      <username>0123456789ABCDEFGHIJ</username>
      <password>0123456789abcdefghijklmnopqrstuvwxyzABCD</password>
    </server>
    ...
  </servers>
  ...
</settings>

Alternatively, the access and secret keys for the account can be provided using

• AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY) environment variables
• aws.accessKeyId and aws.secretKey system properties
• The Amazon EC2 Instance Metadata Service
• AWS-Profile ( Can be overridden with AWS_PROFILE variable )

Using IAM roles

By using environment variables

If users want to / or need to use roles while accessing services, an assumed role can be taken into use by defining two environment variables.

export AWS_ASSUME_ROLE_ARN="ARN:TO:ROLE"
export AWS_ASSUME_ROLE_NAME="session-name-for-role"

By using project specific config file

Alternative way is to create .s3_config file to the project root folder (syntax follows Java .properties file syntax):

# .s3_config
AWS_ASSUME_ROLE_ARN=ARN:TO:ROLE
AWS_ASSUME_ROLE_NAME=session-name-for-role

If you want to use different filename / path for your config file, you can use S3_MAVEN_CONFIG_FILE=<path-to-your-config-file> environment variable.

If you want to disable ASU even though you've set the config file (e.g. in your CI environment), you can override the config variables with empty environment variables

export AWS_ASSUME_ROLE_ARN=
export AWS_ASSUME_ROLE_NAME=

Config precedence

1. Use environment variables if they exist
2. If environment variables don't exist, try to use config file

Making Artifacts Public

This wagon doesn't set an explict ACL for each artfact that is uploaded. Instead you should create an AWS Bucket Policy to set permissions on objects. A bucket policy can be set in the AWS Console and can be generated using the AWS Policy Generator.

In order to make the contents of a bucket public you need to add statements with the following details to your policy:

If your policy is setup properly it should look something like:

{
  "Id": "Policy1397027253868",
  "Statement": [
    {
      "Sid": "Stmt1397027243665",
      "Action": [
        "s3:ListBucket"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::<BUCKET>",
      "Principal": {
        "AWS": [
          "*"
        ]
      }
    },
    {
      "Sid": "Stmt1397027177153",
      "Action": [
        "s3:GetObject"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::<BUCKET>/*",
      "Principal": {
        "AWS": [
          "*"
        ]
      }
    }
  ]
}

If you prefer to use the command line, you can use the following script to make the contents of a bucket public:

BUCKET=<BUCKET>
TIMESTAMP=$(date +%Y%m%d%H%M)
POLICY=$(cat<<EOF
{
  "Id": "public-read-policy-$TIMESTAMP",
  "Statement": [
    {
      "Sid": "list-bucket-$TIMESTAMP",
      "Action": [
        "s3:ListBucket"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::$BUCKET",
      "Principal": {
        "AWS": [
          "*"
        ]
      }
    },
    {
      "Sid": "get-object-$TIMESTAMP",
      "Action": [
        "s3:GetObject"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::$BUCKET/*",
      "Principal": {
        "AWS": [
          "*"
        ]
      }
    }
  ]
}
EOF
)

aws s3api put-bucket-policy --bucket $BUCKET --policy "$POLICY"