Authentication bugfix

[jklinger]

I had been using the Spring S3 wagon (org.springframework.build.aws.maven 3.0.0.RELEASE), but when I try to run the site goal on my projects, it fails complaining about conflicting versions of commons logging. So I switched to your wagon, which simply didn't work (i.e. it didn't fail my build, but it didn't actually connect to S3 either). It kept telling my that the connection was refused with a 403 from AWS. After much debugging, I found the problem in the call to list all the buckets. I think this was failing for me because we've set up our security so that our maven user only has access to our maven repo bucket, but we did it in some (apparently) unusual way using a bucket policy. Without getting too far into the details that I frankly don't understand, my theory is that the account I'm connecting with thinks it has several buckets, although it is in fact forbidden from accessing all but one of them. Thus, listing buckets fails. I looked through the code and could not find any place that actually needed a Bucket instance, and was only using its bucket object to call getName(). I have changed the code to operate on a bucketName directly instead of bucket.getName(). All the tests pass, including the new one I wrote, and the code works for me; however, I did I removed a protected method and I'm not sure if there's some other dependency on it in some other project.

Please let me know if you have any questions; I will be happy to explain more about what I did.

• Justin

[jcaddel]

Thanks for the pull request. I'll take a look at this one. I don't think the call to listBuckets() is required. Essentially all the wagon is doing there is attempting to find a bucket that matches the name provided in the config.

Rearranging the logic so the listBuckets() call is no longer needed seems reasonable.

[yegor256]

I'm having exactly the same problem, and can't use your Wagon

[jklinger]

What is the status of this pull request? I'd really like to start using your wagon.

[jcaddel]

Apologies for the delay here. A careful re-read of the Javadoc for AmazonS3.listBuckets() helped me get a handle on what is going on with this.

If you get a chance could you verify either (or both) of the following?

• The bucket your wagon connects to was created by a different user (same account, just different user in that account)
• The user your wagon connects as does not have permissions to create buckets in your account

If the first is true, the list returned by the AmazonS3.listBuckets() method is not going to contain the desired bucket. This would cause the S3Wagon.getOrCreateBucket() method to fall through it's loop, assume no bucket with that name exists, and try to create a new bucket.

Trying to create a new bucket is then failing due to one of the following reasons:

• The user doesn't have permissions to create a bucket OR
• The bucket already exists

If one of those scenarios rings true, the Javadoc for AmazonS3.listBuckets() could use better wording. It states:

Returns a list of all Amazon S3 buckets that the authenticated sender of the request owns.
...
Anonymous requests cannot list buckets, and users cannot list buckets that they did not create. 

I think this should read:

Returns a list of any Amazon S3 buckets that the authenticated sender of the request created.
...
Anonymous requests cannot list buckets.  Authenticated requests will only include buckets 
that were originally created by the currently authenticated user.

The word "created" is much clearer to me than the word "owns".

At any rate, I don't think the maven-s3-wagon should fail if an S3 bucket was originally created by a different user, as long as it has read/write/list permissions.

It would, however, be good to add an explicit permissions check to make sure the wagon has read/write/list permissions.

All that being said, I should be able to get your pull request (or equivalent thereof) merged in and released at some point today.

[jcaddel]

Just published version 1.1.15 which fixes this. It is available from the Kuali Maven Repository right now, and should be out on Maven Central within a few hours.

Thanks again for the pull request, I used a slightly modified version of what you provided.

This update makes it so that the maven-s3-wagon no longer needs to "own" (aka "be the creator of") the bucket it is accessing.

As long as the wagon has read/write/list permissions on the bucket everything should work.

For what it is worth, this exposed an (apparent) shortcoming in the AWS SDK for Java. There does not seem to be an easy way to determine what permissions the currently logged in user has on a bucket. The AWS forums suggest performing an operation in order to determine if you have permission to do that operation.

Ideally, I'd like to be able to ask if I have a permission (for example to upload a file) without having to try and actually upload a file.

If anyone has a simple method for discovering bucket permissions ahead of time, I'd love to hear about it.