search

embl-cba / imagej-utils

BSD 2-Clause "Simplified" License
2 stars 4 forks source link

Add support for amazon s3 in FileAndUrlUtils #36

Closed constantinpape closed 3 years ago

constantinpape commented 3 years ago

The exists and getInputStream methods are working. I am not quite sure what to do in getParentLocation. Should it be the whole parent url or just the parent in the bucket? @tischi could you point me to some example usage of that method?

constantinpape commented 3 years ago

Also, this only works for buckets with Anon credentials right now. To support real authentication, we would need something similar to https://github.com/mobie/mobie-viewer-fiji/blob/master/src/main/java/de/embl/cba/mobie/n5/S3CredentialsCreator.java We may consider refactoring this to imagej utils?

tischi commented 3 years ago

We may consider refactoring this to imagej utils?

Yes, makes sense!

constantinpape commented 3 years ago

@tischi I had to make some more changes to enable loading tables from s3 as well, but now everything I tested (opening a project, adding sources, adding table and loading columns and adding bookmarks) works.

For now, this only works for buckets that have Anonymous access; to support access with credentials, the authentication here would need to be exposed as parameter. Or it might be possible to find out the correct authentication without passing the argument; I want to try that in the next couple of days, but I will first need to set up an example project on our s3 with credentials.

Anyway, I think this can be merged now and the changes to the authentication can be done in a separate PR.

tischi commented 3 years ago

Anyway, I think this can be merged now and the changes to the authentication can be done in a separate PR.

Yes, I agree, let's deal with the authentication later as this may be a rabbit hole.