Profile Support Not Working

mjallday commented 8 years ago

Using the AWS credentials directly works

`AWS_ACCESS_KEY_ID=XXX AWS_SECRET_ACCESS_KEY=YYY mvn -f site-pom.xml deploy`

``` [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Very Good Parent POM 0.0.1-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ vg-parent-pom --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /Users/marshall/code/vg/commons-data/src/main/resources [INFO] [INFO] --- maven-compiler-plugin:3.3:compile (default-compile) @ vg-parent-pom --- [INFO] No sources to compile [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ vg-parent-pom --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /Users/marshall/code/vg/commons-data/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:3.3:testCompile (default-testCompile) @ vg-parent-pom --- [INFO] No sources to compile [INFO] [INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ vg-parent-pom --- [INFO] No tests to run. [INFO] [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ vg-parent-pom --- [WARNING] JAR will be empty - no content was marked for inclusion! [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ vg-parent-pom --- [INFO] Installing /Users/marshall/code/vg/commons-data/target/vg-parent-pom-0.0.1-SNAPSHOT.jar to /Users/marshall/.m2/repository/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-SNAPSHOT.jar [INFO] Installing /Users/marshall/code/vg/commons-data/site-pom.xml to /Users/marshall/.m2/repository/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-SNAPSHOT.pom [INFO] [INFO] --- maven-deploy-plugin:2.7:deploy (default-deploy) @ vg-parent-pom --- Downloading: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/maven-metadata.xml Uploading: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-20160714.234542-1.jar Uploaded: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-20160714.234542-1.jar (4 KB at 2.4 KB/sec) Uploading: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-20160714.234542-1.pom Uploaded: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-20160714.234542-1.pom (11 KB at 7.6 KB/sec) Downloading: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/maven-metadata.xml Uploading: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/maven-metadata.xml Uploaded: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/maven-metadata.xml (775 B at 0.5 KB/sec) Uploading: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/maven-metadata.xml Uploaded: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/maven-metadata.xml (285 B at 0.3 KB/sec) [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 9.683 s [INFO] Finished at: 2016-07-14T16:45:47-07:00 [INFO] Final Memory: 19M/304M [INFO] ------------------------------------------------------------------------ ```

Using AWS CLI with the profile works

AWS_PROFILE=vg aws s3 ls s3://very-good/software/snapshot/
                           PRE com/
2016-07-14 16:45:48          0

Using the profile does not work

`AWS_PROFILE=vg mvn -f site-pom.xml deploy`

``` [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Very Good Parent POM 0.0.1-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ vg-parent-pom --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /Users/marshall/code/vg/commons-data/src/main/resources [INFO] [INFO] --- maven-compiler-plugin:3.3:compile (default-compile) @ vg-parent-pom --- [INFO] No sources to compile [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ vg-parent-pom --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /Users/marshall/code/vg/commons-data/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:3.3:testCompile (default-testCompile) @ vg-parent-pom --- [INFO] No sources to compile [INFO] [INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ vg-parent-pom --- [INFO] No tests to run. [INFO] [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ vg-parent-pom --- [WARNING] JAR will be empty - no content was marked for inclusion! [INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ vg-parent-pom --- [INFO] Installing /Users/marshall/code/vg/commons-data/target/vg-parent-pom-0.0.1-SNAPSHOT.jar to /Users/marshall/.m2/repository/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-SNAPSHOT.jar [INFO] Installing /Users/marshall/code/vg/commons-data/site-pom.xml to /Users/marshall/.m2/repository/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/vg-parent-pom-0.0.1-SNAPSHOT.pom [INFO] [INFO] --- maven-deploy-plugin:2.7:deploy (default-deploy) @ vg-parent-pom --- Downloading: s3://very-good/software/snapshot/com/verygood/vg-parent-pom/0.0.1-SNAPSHOT/maven-metadata.xml [INFO] Skip unsupported property name role_arn in profile [vault-prod]. [INFO] Skip unsupported property name source_profile in profile [vault-prod]. [INFO] Skip unsupported property name mfa_serial in profile [vault-prod]. [WARNING] Could not transfer metadata com.verygood:vg-parent-pom:0.0.1-SNAPSHOT/maven-metadata.xml from/to vg-snapshot (s3://very-good/software/snapshot/): Unable to load AWS credentials from any provider in the chain [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 4.897 s [INFO] Finished at: 2016-07-14T16:50:33-07:00 [INFO] Final Memory: 15M/245M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy (default-deploy) on project vg-parent-pom: Failed to retrieve remote metadata com.verygood:vg-parent-pom:0.0.1-SNAPSHOT/maven-metadata.xml: Could not transfer metadata com.verygood:vg-parent-pom:0.0.1-SNAPSHOT/maven-metadata.xml from/to vg-snapshot (s3://very-good/software/snapshot/): Unable to load AWS credentials from any provider in the chain -> [Help 1] [ERROR] ```

Extension is installed via

        <extensions>
            <extension>
                <groupId>fi.yle.tools</groupId>
                <artifactId>aws-maven</artifactId>
                <version>1.4.0</version>
            </extension>
        </extensions>

and

    <pluginRepositories>
        <pluginRepository>
            <id>yle-public</id>
            <name>Yle public repository</name>
            <url>http://maven.c4.yle.fi/release</url>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </pluginRepository>
    </pluginRepositories>

Any ideas here?

mjallday commented 8 years ago

Update - Turns out I have some profiles in ~/.aws/credentials which use IAM based cross account signin and this version of the AWS SDK does not handle this well.

You can see this in the above output where it gives the message [INFO] Skip unsupported property name role_arn in profile [vault-prod].. Upgrading AWS SDK should resolve that. Any chance you guys will do that? Feel free to close this issue if not.

huima commented 8 years ago

It is currently vacation season here, but will check this in few weeks - when back at the office, unless Matti has spare time already.

milankinen commented 8 years ago

@mjallday I'm not sure if I understood the big picture, so sorry about this recap. So if you try directly by using access key and secret it works but if you use profile (which contains the key and secret) it doesn't, right?

Hmmm... I'm using this tool daily with AWS_PROFILE and it's working fine. What kind of error message do you get when you try to run deploy with profile? Also, what do you mean with that "IAM based cross account signin"?

hopel commented 8 years ago

@milankinen If I understand the last comment by @mjallday correctly, he is trying to use a profile that does not contain aws_access_key_id and aws_secret_access_key entries, but role_arn and source_profile entries instead, i.e. a profile entry to assume a role (based on the credentials of the source_profile).

Support for this type of profile got added to the AWS Java SDK with release 1.10.63:

Support for using assume roles in the shared credentials file much like the CLI.

As this project is currently using the AWS Java SDK version 1.10.59, updating that should automatically add the missing support for the new profile type.

Yleisradio / aws-maven

Profile Support Not Working #6