Open xinlaoda opened 3 years ago
The S3 requests must be sent to the destination region. As in this case AWS Data Replication Hub have configured default region to be the source region, It will need to do an override to destination region, for example:
From EC2 in SG, ap-southeast-1, do following s3 cp CLI:
$ aws s3 cp s3://xxin-package-af-south-1/cm6/x86_64/cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm /tmp download failed: s3://xxin-package-af-south-1/cm6/x86_64/cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm to ../../tmp/cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm An error occurred (IllegalLocationConstraintException) when calling the GetObject operation: The af-south-1 location constraint is incompatible for the region specific endpoint this request was sent to.
If using "--region
Hi @xinlaoda
The cause of your failure is because you deployed the solution in ap-southeast-1 and this tool assumed the region of the source bucket is ap-southeast-1.
We can't support the deployment directly in af-south-1 or me-south-1 for some reason.
However, you can still use this solution to replicate data from those two regions to China regions. The suggestion is that you can deploy this tool in China regions directly and set the source region as those two. If you have some issues deploying data replication hub in China regions like you don't have a ICP licensed domain name etc, you can directly deploy this plugin (follow the readme of this repo) instead.
Hope this clarifies, Thanks for using this solution.
Two issues & comments:
make sense. we will upgrade to support af-south-1 and me-south-1 regions.
Describe the bug when copy S3 objects from af-south-1, error happens.
To Reproduce
Expected behavior Support af-south-1 and me-south-1
Screenshots ==== CloudWatch Log ====
INFO:JobSender> Start comparing...
Traceback (most recent call last): File "job_sender.py", line 105, in
find_and_send_jobs(src_client, des_client, sqs_queue_name,
File "job_sender.py", line 32, in find_and_send_jobs
job_sender.send_jobs(include_version)
File "/app/migration_lib/job.py", line 122, in send_jobs
self._get_delta_and_send(include_version)
File "/app/migration_lib/job.py", line 83, in _get_delta_and_send
src_file_set = self._get_source_set(src_list_gen, include_version)
File "/app/migration_lib/job.py", line 38, in _get_source_set
src_list = next(src_list_gen)
File "/app/migration_lib/client.py", line 218, in _list_objects_without_version
response = self._list_objects_func(list_kwargs)
File "/app/migration_lib/client.py", line 202, in _list_objects_func
return self._client.list_objects_v2(kwargs)
File "/usr/local/lib/python3.8/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.8/site-packages/botocore/client.py", line 676, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (IllegalLocationConstraintException) when calling the ListObjectsV2 operation: The af-south-1 location constraint is incompatible for the region specific endpoint this request was sent to.
Please complete the following information about the solution:
Additional context Add any other context about the problem here.