search

irods / irods_resource_plugin_s3

S3-compatible storage resource plugin for iRODS
Other
12 stars 16 forks source link

Cacheless: Document AWS S3 V4 via S3_SIGNATURE_VERSION #1875

Open JustinKyleJames opened 5 years ago

JustinKyleJames commented 5 years ago

Currently AWS S3 V4 is not working properly due to the path being incorrect when calculating the signature.

serverhorror commented 5 years ago

I have the same problem

bitkid commented 5 years ago

is there an ETA when this will make it to master? it's kinda critical to us.

trel commented 5 years ago

@bitkid this issue is for the cacheless codepath (which has not yet been released).

The existing S3 V4 works today, as far as I know.

bitkid commented 5 years ago

do i have to explicitly define the auth algorithm somewhere? in the log file i see

Jun 18 14:19:13 pid:181 remote addresses: 10.4.81.96 ERROR: S3Status: [ErrorInvalidRequest] - 78 Jun 18 14:19:13 pid:181 remote addresses: 10.4.81.96 ERROR: S3Host: s3.eu-central-1.amazonaws.com Jun 18 14:19:13 pid:181 remote addresses: 10.4.81.96 ERROR: Function: responseCompleteCallback Jun 18 14:19:13 pid:181 remote addresses: 10.4.81.96 ERROR: Message: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256. Jun 18 14:19:13 pid:181 remote addresses: 10.4.81.96 ERROR: Extra Details: Jun 18 14:19:13 pid:181 remote addresses: 10.4.81.96 ERROR: RequestId: 9ECC550F003DE034 Jun 18 14:19:13 pid:181 remote addresses: 10.4.81.96 ERROR: HostId: 86s7mGXIyRWAt5Op9J1lmUUHgBqZiIPphz4vX2vCsTbpjlvOmb5zZS2wvWueHVnKYeLcIlW2MRk= Jun 18 14:19:16 pid:181 remote addresses: 10.4.81.96 ERROR: [-] /tmp/tmp919rQK/s3/libirods_s3.cpp:2324:irods::error s3SyncToArchPlugin(irods::plugin_context &, const char *) : status [S3_PUT_ERROR] errno [] -- message [s3PutCopyFile - Error putting the S3 object: "/mybucket/resource1/data/home/irodsAdmin/VERSION.json" - "ErrorInvalidRequest"]

bitkid commented 5 years ago

hmm .. maybe setting S3_SIGNATURE_VERSION would help. should probably be in the docs somewhere (incl valid values)

JustinKyleJames commented 5 years ago

Set the S3_SIGNATURE_VERSION in the context string to either "2" or "4".

We have an issue to improve documentation and will be doing so soon.