search

wso2 / micro-integrator

The cloud-native configuration driven runtime that helps developers implement composite microservices.
https://wso2.com/integration/
Apache License 2.0
196 stars 208 forks source link

[AWS S3 Connector] : AWS S3 Connector 1.0.9 failed with Invalid x-amz-date format #1219

Open preetam-patil opened 4 years ago

preetam-patil commented 4 years ago

Description:

Amazon S3 connector 1.0.9 returns invalid date format error while trying GetObject method

“Invalid date format header, expected to be in ISO8601, RFC1123 or RFC1123Z time format”

Suspected Issue: x-amz-date header returns invalid format "x-amz-date: Wed, 11 Mar 2020 16:42:18 GMT[\r][\n]"

Affected Product Version: AWS-S3 Connector 1.0.9

OS, DB, other environment details and versions:
MI-1.1.0 , CentOS

Steps to reproduce: a) Deploy AWS S3 connector 1.0.9 to MI-1.1.0 and trigger getObject method with below configuration

`

xxxxxxxxxxxxx 
<secretAccessKey>xxxxxxxxxxxxx</secretAccessKey>
<region>us-east-1</region>
<methodType>GET</methodType>
<addCharset>false</addCharset>
<isXAmzDate>true</isXAmzDate>
<xAmzAcl>public-read-write</xAmzAcl>
<addCharset>false</addCharset>

</amazons3.init>

xxxxxxxxxxxxx inbox/test.txt application/octet-stream UTF-16 ` **Possible Solution:** The time stamp must be in UTC and in the following ISO 8601 format: YYYYMMDD'T'HHMMSS'Z'. For example, 20150830T123600Z is a valid time stamp. Do not include milliseconds in the time stamp. **Reference: https://docs.amazonaws.cn/en_us/general/latest/gr/sigv4-date-handling.html** **Detailed Logs:** [2020-03-11 17:42:18,890] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 << "GET /testbucket/inbox/test.txt?response-content-type=application/octet-stream&response-content-encoding=UTF-16 HTTP/1.1[\r][\n]" [2020-03-11 17:42:18,890] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 << "Authorization: AWS4-HMAC-SHA256 Credential=admin/20200311/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-acl;x-amz-content-sha256;x-amz-date,Signature=f80cf27517f2df15a8cd32d38319c96f3a2d0025da724fbaec596c0e958a8577[\r][\n]" [2020-03-11 17:42:18,890] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 << "x-amz-content-sha256: UNSIGNED-PAYLOAD[\r][\n]" [2020-03-11 17:42:18,890] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 << "x-amz-acl: public-read-write[\r][\n]" [2020-03-11 17:42:18,891] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 << **"x-amz-date: Wed, 11 Mar 2020 16:42:18 GMT[\r][\n]"** [2020-03-11 17:42:18,891] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 << "Content-Type: application/xml[\r][\n]" [2020-03-11 17:42:18,891] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 << "Host: xxxxxxxxxxxxx[\r][\n]" [2020-03-11 17:42:18,891] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 << "Connection: Keep-Alive[\r][\n]" [2020-03-11 17:42:18,891] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 << "User-Agent: Synapse-PT-HttpComponents-NIO[\r][\n]" [2020-03-11 17:42:18,892] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 << "[\r][\n]" [2020-03-11 17:42:18,892] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 >> "HTTP/1.1 400 Bad Request[\r][\n]" [2020-03-11 17:42:18,892] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 >> "Accept-Ranges: bytes[\r][\n]" [2020-03-11 17:42:18,892] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 >> "Content-Length: 385[\r][\n]" [2020-03-11 17:42:18,893] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 >> "Content-Security-Policy: block-all-mixed-content[\r][\n]" [2020-03-11 17:42:18,893] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 >> "Content-Type: application/xml[\r][\n]" [2020-03-11 17:42:18,893] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 >> "Server: MinIO/RELEASE.2020-03-09T18-26-53Z[\r][\n]" [2020-03-11 17:42:18,893] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 >> "Vary: Origin[\r][\n]" [2020-03-11 17:42:18,893] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 >> "X-Amz-Request-Id: 15FB4D87A2505270[\r][\n]" [2020-03-11 17:42:18,894] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 >> "X-Xss-Protection: 1; mode=block[\r][\n]" [2020-03-11 17:42:18,894] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 >> "Date: Wed, 11 Mar 2020 16:42:18 GMT[\r][\n]" [2020-03-11 17:42:18,894] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 >> "[\r][\n]" [2020-03-11 17:42:18,894] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 >> "[\n]" [2020-03-11 17:42:18,895] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-1 >> "MalformedDateInvalid date format header, expected to be in ISO8601, RFC1123 or RFC1123Z time format.inbox/test.txttestbucket/testbucket/inbox/test.txt15FB4D87A2505270eb83867a-334a-495f-9a22-9118656f401c" [2020-03-11 17:42:18,944] INFO {org.apache.synapse.mediators.builtin.LogMediator} - To: http://www.w3.org/2005/08/addressing/anonymous, WSAction: , SOAPAction: , MessageID: urn:uuid:9f6b4a37-fd98-4877-9f8c-9f109c6b24fe, Direction: request, STATUS = Get Object Complete..........., Envelope: **MalformedDateInvalid date format header, expected to be in ISO8601, RFC1123 or RFC1123Z time format.**inbox/test.txttestbucket/testbucket/inbox/test.txt15FB4D87A2505270eb83867a-334a-495f-9a22-9118656f401c
sdkottegoda commented 4 years ago

Hi @preetam-patil, Thank you for reporting this issue. The Amazon S3 connector is not yet compatible with Micro Integrator and it should fail at the startup itself. We are working on fixing the issue and we will be releasing a new connector version once the issues are fixed. Please refer to https://github.com/wso2/micro-integrator/issues/1128 for more information.

preetam-patil commented 4 years ago

Hi WSO2 Team, Just a Suggestion: Most of the issues with WSO2 AWS related Connectors are related to java implementation of REST API from AWS. It would be great if we align our AWS connectors using aws-java-sdk directly so that we can re-use aws code which would also avoid reinventing all rest-api implementations by WSO2 team and fixing issues as aws keeps changing. Reference: https://github.com/aws/aws-sdk-java

In other words, can we just not write a connector on top of the aws-sdk-java ?