Roundtrip to S3 fails with Dynamic Revisions

[aecollver]

Observed Behavior

• I can resolve a specific version of a module published to S3 with sbt-s3-resolver.
• I can't resolve using dynamic revisions (e.g. 1.0.+) of a module published to S3 with sbt-s3-resolver.
• I can resolve dynamic revisions (e.g. 1.0.+) of a module published to S3 with another plugin: https://github.com/frugalmechanic/fm-sbt-s3-resolver

  Cause

I found that the fm-sbt-s3-resolver is creating a "directory" object in S3 (an empty object that has a key ending in "/"). Creating the same "directory" object from the AWS console allows Ivy to resolve a module published to S3 using sbt-s3-resolver.

Ivy's FileRepository returns a mix of directory and file names (but one level deep unlike the S3 listObjects overload that is currently used): https://github.com/sbt/ivy/blob/2.3.x-sbt/src/java/org/apache/ivy/plugins/repository/file/FileRepository.java#L99-L112

Proposed Fix

Update https://github.com/ohnosequences/ivy-s3-resolver/blob/master/src/main/java/ohnosequences/ivy/S3Repository.java#L114-L133

• Use this listObjects overload: http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3Client.html#listObjects(com.amazonaws.services.s3.model.ListObjectsRequest)
• Specify delimiter "/": http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/ListObjectsRequest.html#getDelimiter()
• Return a combination of objects (files) and common prefixes (directories) similar to Ivy's FileRepository.

[aecollver]

Here's an untested implementation of the proposed fix:

https://github.com/acollver/ivy-s3-resolver/commit/98142489ea099e73330b75423127480615392a96

[aecollver]

I have verified that my proposed fix works with both specific and dynamic versions.

[aecollver]

I created a pull request for this issue. Apologies for creating the issue against sbt-s3-resolver when the proposed fix belongs in ivy-s3-resolver:

https://github.com/ohnosequences/ivy-s3-resolver/pull/7

[aecollver]

I updated my pull request to support pagination with AmazonS3.listObject

[laughedelic]

Hi @acollver! Thanks a lot! Actually, I wanted this feature too, but didn't know how to implement it and didn't have time to investigate. The ivy-s3-resolver is maintained by @evdokim, but he's too busy now to review the code, so I'll just merge it and release a new version.

[laughedelic]

I just released the new version with the fix: v0.12.0.