Describe the bug
Run the following code to transfer 15GB data from one source to two destination buckets in the same region aws:us-east-1, the transfer fails
2023-05-09T03:17:56.595177698Z 03:17:56 [ERROR] Exception: An error occurred (NoSuchUpload) when calling the
2023-05-09T03:17:56.595192951Z UploadPart operation: The specified upload does not exist. The upload ID may be
2023-05-09T03:17:56.595194689Z invalid, or the upload may have been aborted or completed.
Describe the bug Run the following code to transfer 15GB data from one source to two destination buckets in the same region
aws:us-east-1
, the transfer failsGateway Log
This should be because currently the multipart upload ID is initialized per region. It should be done per-region per-bucket per-key: https://github.com/skyplane-project/skyplane/blob/54f8a4cee2e2149ab25352460620a1958a095316/skyplane/api/transfer_job.py#L114