aws-solutions / data-transfer-hub

Seamless User Interface for replicating data into AWS.
https://aws-solutions.github.io/data-transfer-hub/en/
Apache License 2.0
136 stars 22 forks source link

Task execution Error, 400 error reported #104

Closed Fabulous-maker closed 1 year ago

Fabulous-maker commented 1 year ago

After I have deployed all the relevant configurations, the following 400 error reports appear. The same problem still exists after the redeployment. The format set in SecretManager is as follows: { "access_key_id": "", "secret_access_key": "" }

400 error messages:"api error invalidToken:The provided token is malformed or otherwise invalid"

JoeShi commented 1 year ago

Hi there, Have you tried to use the credentials with AWS CLI to test if it is still valid? Have your copied the format directly from https://awslabs.github.io/data-transfer-hub/en/faq/#error-messages?

Fabulous-maker commented 1 year ago

Thank you for your reply

After an error is reported, I locally tested the permissions of the given Global AKSK, which has the permissions to list objects and get objects, and the format in the SecretManager is directly copied, as follows:

{ "access_key_id": "AKIAZxxxxxxxxx3W5", "secret_access_key": "rRRpexxxxxxxxxxxxxrfge8Jmo2" }

The configuration steps follow the steps in the document:https://awslabs.github.io/data-transfer-hub/en/deployment/

Is it caused by my configuration error? I also deployed through Auth0, but after the task is created, it will directly change from the Starting state to the Error state. I queried the relevant Cloudwatch Logs and did not record the detailed error information.

YikaiHu commented 1 year ago

Hi @Fabulous-maker , what's your version of Data Transfer Hub? You can find the version in CloudFormation Stack Info Description.

And could you check the step function log whose function name is like CfnWorkflowCfnDeployment?

Fabulous-maker commented 1 year ago

Template version v1.0.1,Is it because the version is too low? However, 400 errors will occur for resources deployed through Authoring(Latest template Version 2.3.0--Last updated: 12/2022).

I launch template in this link:https://github.com/awslabs/data-transfer-hub/blob/main/docs/openId/auth0/DTH_Auth0.md

image (3)

evalzy commented 1 year ago

Template version v1.0.1,Is it because the version is too low? However, 400 errors will occur for resources deployed through Authoring(Latest template Version).

I launch template in this link:https://github.com/awslabs/data-transfer-hub/blob/main/docs/openId/auth0/DTH_Auth0.md

image (3)

Hi @Fabulous-maker , we are working on the investment and debugging, will update later.

Fabulous-maker commented 1 year ago

Thank you for your update.

I have redeployed the latest version(2.3.0) of the template. There is still 400 error reported.

It can operate normally when transferring bucket objects in China. However, 400 errors will be reported when synchronizing global buckets to Chinese buckets.

At present, I am testing the task operation from China to Global region. 400 Error still reported.

Please let me know if you need any information or what operation I need.

The following is the error message in Cloudwatch logs:

image (4)

YikaiHu commented 1 year ago

The 400 error code is related to credential key.

Could you please create a new AWS Credential key and stored in a new Sercets Manager's Secret? You need to create a new transfer task.

Fabulous-maker commented 1 year ago

Thanks for your update. I will test later and post test'results in this issue

JoeShi commented 1 year ago

just quick check. have you removed the < and > within the " "?

Fabulous-maker commented 1 year ago

Thank you for your reply.

I have removed these two special characters(< >) when adding AKSK. And according to @YikaiHu suggestion, I reapplied for a new AKSK in the Global region, and the created task can run normally.

However, the AKSK created before will still report errors.

I later updated the AKSK I created before, and 400 errors will still occur.

That is to say, two keys A and B are stored in the SecretManager. The template was not deployed when A was created, and B was deployed after creation. But now the AKSK in A and B are the same. A will report an error while B will run normally. It's strange.

image (3) image (2) image (1) image

YikaiHu commented 1 year ago

Hi @Fabulous-maker , we found the root cause, you put an extra space at the end of access_key_id

image