使用aws s3创建数据源时报错

[lxlong0103]

Issue Type

Running

Have you searched for existing documents and issues?

Yes

OS Platform and Distribution

ubentu 22.04

All_in_one Version

secretpad:0.8.1b0

Kuscia Version

kuscia:0.9.0b0

What happend and What you expected to happen.

在edge节点，登录进页面之后，数据源管理-》注册数据源-〉OSS

在填写endpoint时，填入：http://s3.cn-north-1.amazonaws.com.cn。看代码里校验了http/https

其他参数已校验可以正常工作。

报错：S3 API请求与endpoint端口不匹配。

看代码中region是空的，报错是region不能为空：               
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(
                        config.getEndpoint(),
                        "")) // 这里的region是空的

Log output.

2024-07-03 10:33:04 [http-nio-8080-exec-4] INFO  o.s.secretpad.service.util.HttpUtils - detection url http://s3.cn-north-1.amazonaws.com.cn
2024-07-03 10:33:04 [http-nio-8080-exec-4] INFO  o.s.s.service.impl.OssServiceImpl - start check bucket
2024-07-03 10:33:04 [http-nio-8080-exec-4] ERROR o.s.s.service.impl.OssServiceImpl - There was an error response from S3. This might indicate that the credentials are invalid.
2024-07-03 10:33:04 [http-nio-8080-exec-4] ERROR o.s.s.service.impl.OssServiceImpl - Error Message:    <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; a non-empty region must be provided in the credential.</Message><RequestId>SYP82MJ0XFZY9HC4</RequestId><HostId>JZpMT2TnVI8Q/+NAZznH0rKOYDLoU4PNbqhDG915cm8mZsmSCz0eXaGDuSCCvyVejKnJg33RtHc=</HostId></Error>
2024-07-03 10:33:04 [http-nio-8080-exec-4] ERROR o.s.s.w.e.SecretpadExceptionHandler - find SecretpadException error: DATA_SOURCE_ENDPOINT_API_PORT_ERROR, message: The authorization header is malformed; a non-empty region must be provided in the credential. (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: SYP82MJ0XFZY9HC4; S3 Extended Request ID: JZpMT2TnVI8Q/+NAZznH0rKOYDLoU4PNbqhDG915cm8mZsmSCz0eXaGDuSCCvyVejKnJg33RtHc=; Proxy: null)
org.secretflow.secretpad.common.exception.SecretpadException: The authorization header is malformed; a non-empty region must be provided in the credential. (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: SYP82MJ0XFZY9HC4; S3 Extended Request ID: JZpMT2TnVI8Q/+NAZznH0rKOYDLoU4PNbqhDG915cm8mZsmSCz0eXaGDuSCCvyVejKnJg33RtHc=; Proxy: null)

[aokaokd]

好的，收到您的问题，这个看上去是亚马逊的数据源。 辛苦您看下这个地方传入的值是否正确

[zimu-yuxi]

Issue Type

Running

Have you searched for existing documents and issues?

Yes

OS Platform and Distribution

ubentu 22.04

All_in_one Version

secretpad:0.8.1b0

Kuscia Version

kuscia:0.9.0b0

What happend and What you expected to happen.

在edge节点，登录进页面之后，数据源管理-》注册数据源-〉OSS

在填写endpoint时，填入：http://s3.cn-north-1.amazonaws.com.cn。看代码里校验了http/https

其他参数已校验可以正常工作。

报错：S3 API请求与endpoint端口不匹配。

看代码中region是空的，报错是region不能为空：               
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(
                        config.getEndpoint(),
                        "")) // 这里的region是空的

Log output.

2024-07-03 10:33:04 [http-nio-8080-exec-4] INFO  o.s.secretpad.service.util.HttpUtils - detection url http://s3.cn-north-1.amazonaws.com.cn
2024-07-03 10:33:04 [http-nio-8080-exec-4] INFO  o.s.s.service.impl.OssServiceImpl - start check bucket
2024-07-03 10:33:04 [http-nio-8080-exec-4] ERROR o.s.s.service.impl.OssServiceImpl - There was an error response from S3. This might indicate that the credentials are invalid.
2024-07-03 10:33:04 [http-nio-8080-exec-4] ERROR o.s.s.service.impl.OssServiceImpl - Error Message:    <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; a non-empty region must be provided in the credential.</Message><RequestId>SYP82MJ0XFZY9HC4</RequestId><HostId>JZpMT2TnVI8Q/+NAZznH0rKOYDLoU4PNbqhDG915cm8mZsmSCz0eXaGDuSCCvyVejKnJg33RtHc=</HostId></Error>
2024-07-03 10:33:04 [http-nio-8080-exec-4] ERROR o.s.s.w.e.SecretpadExceptionHandler - find SecretpadException error: DATA_SOURCE_ENDPOINT_API_PORT_ERROR, message: The authorization header is malformed; a non-empty region must be provided in the credential. (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: SYP82MJ0XFZY9HC4; S3 Extended Request ID: JZpMT2TnVI8Q/+NAZznH0rKOYDLoU4PNbqhDG915cm8mZsmSCz0eXaGDuSCCvyVejKnJg33RtHc=; Proxy: null)
org.secretflow.secretpad.common.exception.SecretpadException: The authorization header is malformed; a non-empty region must be provided in the credential. (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: SYP82MJ0XFZY9HC4; S3 Extended Request ID: JZpMT2TnVI8Q/+NAZznH0rKOYDLoU4PNbqhDG915cm8mZsmSCz0eXaGDuSCCvyVejKnJg33RtHc=; Proxy: null)

根据报错：a non-empty region must be provided in the credential，确认下credential是否提供了AK和SK，

[lxlong0103]

AK和SK是正确的。且正在用着。

[zimu-yuxi]

可以提供下创建数据源的入参吗？比如F12，给个入参的截图或者copy出来，感谢

[lxlong0103]

ak和sk是生产环境用的，不能公开，已隐掉部分字符。

curl 'http://1.1.1.1:8089/api/v1alpha1/datasource/create' \ -H 'Accept: application/json' \ -H 'Accept-Language: zh-CN,zh;q=0.9,en;q=0.8' \ -H 'Connection: keep-alive' \ -H 'Content-Type: application/json' \ -H 'Origin: http://1.1.1.1:8089' \ -H 'Referer: http://1.1.1.1:8089/node?nodeId=ucivrxxm&tab=data-source' \ -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36' \ -H 'User-Token: 436ac37415ab413c9d99f1960d5b5e77' \ --data-raw '{"type":"OSS","name":"s3","dataSourceInfo":{"endpoint":"http://s3.cn-north-1.amazonaws.com.cn","ak":"XXXXXDB43V2JQEXXXXXX","sk":"xxxxxxWuzTAJ3ab2Jc7XXXXXM28yB8HUVlxxxxxx","bucket":"custom-data","prefix":"test/secretpad/"},"nodeId":"ucivrxxm"}' \ --insecure

======== response： {"status":{"code":202012507,"msg":"S3 API请求与endpoint端口不匹配."},"data":null}

[zimu-yuxi]

确认endpoint是否是正确的。 我们内部，单测看一下