abashev / vfs-s3

Amazon S3 driver for Apache commons-vfs (Virtual File System) project
Apache License 2.0
93 stars 50 forks source link

Update the samples. #38

Closed ptahchiev closed 7 years ago

ptahchiev commented 7 years ago

The S3Shell in the samples module has this code:

        FileSystemOptions opts = S3FileProvider.getDefaultFileSystemOptions();

however this method is now missing in S3FileProvider. Please update the samples - how are we supposed to know what has replaced this method.

abashev commented 7 years ago

S3Shell?

ptahchiev commented 7 years ago

https://github.com/abashev/vfs-s3/blob/branch-2.4.x/samples/src/com/intridea/io/vfs/samples/S3Shell.java

abashev commented 7 years ago

😳 I totally forgot about these samples

abashev commented 7 years ago

Done

ptahchiev commented 7 years ago

Hey man, I still don't understand how this works :( If i try to use just the FileSystemManager then I get:

com.amazonaws.AmazonClientException: Unable to load AWS credentials from any provider in the chain
    at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:131)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3798)
    at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1078)
    at com.github.vfss3.S3FileSystem.doesBucketExist(S3FileSystem.java:110)
abashev commented 7 years ago

You need to specify credentials with any way from this doc http://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html Usually I use environment variables and http://direnv.net/ for storing it into local file

ptahchiev commented 7 years ago

Well, I'm using spring-boot and my credentials are stored in application.properites file and it was very easy to do this before:

        if (s3Properties.getAccessKey() != null) {
            final StaticUserAuthenticator userAuthenticator = new StaticUserAuthenticator(null, s3Properties.getAccessKey(), s3Properties.getSecretKey());
            final FileSystemOptions options = S3FileProvider.getDefaultFileSystemOptions();
            DefaultFileSystemConfigBuilder.getInstance().setUserAuthenticator(options, userAuthenticator);
        }

I guess I have to use this now:

System.setProperty("aws.accessKeyId", "");
abashev commented 7 years ago

Petar, to be fair it is really bad practice to store any credentials inside your sources - first of all you have to control codebase for accessing, after that you need to control build machine and deployment itself. So best way to turn on IAM roles on your servers or provide all credentials in run-time via environment or system properties. Or you could use file ~/.aws/credentials for all your tests and runs

svella commented 7 years ago

Alexey,

I just discovered this is biting me too. We store credentials securely in a database use StaticUserAuthenticator to pass those credentials in when resolving to a bucket and must have a way to potentially specify credentials on a file by file basis because they may come from different buckets and may have different credentials. Using just the default AWS credential chain of system properties, environment variables, and configuration in the user home directory is too much of a shotgun approach to be viable and the credentials coming from a StaticUserAuthenticator (or other implementations of that interface) need to be injected at the head of the chain when present. I can work on a patch in the next few days to restore this functionality since I need it fairly soon.

abashev commented 7 years ago

@svella nothing was changed for you in this case - you could do something like that S3FileProvider.setDefaultClient(new AmazonS3Client(new BasicAWSCredentials("access", "secret"))) if you need something system-wide or create new file system with specified options and S3Client inside it