search

ExpediaGroup / circus-train

Circus Train is a dataset replication tool that copies Hive tables between clusters and clouds.
Apache License 2.0
86 stars 15 forks source link

Updates for S3S3copier retry #122

Closed max-jacobs closed 5 years ago

max-jacobs commented 5 years ago

Original issue issue 56.

This update addresses a few issues with the s3 to s3 copy retry mechanism added in pull request 119:

  1. The configurable option for the number of max number of attempts has been renamed to s3s3-retry-max-copy-attempts for consistency with other options
  2. The initial and max retry intervals have been increased
  3. AmazonClientException is now being caught when Copy.waitForCompletion() is called to help clarify if the bug occurs here
coveralls commented 5 years ago

Coverage Status

Coverage increased (+0.04%) to 74.904% when pulling 0a151ea36745050300f94c88fda02297c9177cf2 on issue-56-2 into fea303355c2c95014004a381a437c7bcec562017 on master.