skyplane-project / skyplane

🔥 Blazing fast bulk data transfers between any cloud 🔥
https://skyplane.org
Apache License 2.0
1.09k stars 62 forks source link

[bug] Transfer verification error for multipart upload #733

Closed sarahwooders closed 1 year ago

sarahwooders commented 1 year ago

Describe the bug I am transferring a large 135 GB file from AWS to GCP, however instead a 62.5 GB file appears in GS and the transfer validation failes.

To Reproduce Run skyplane cp s3://glisten-datasets/allparel-images/ILSVRC2012_img_train.tar gs://glisten-dataset/allparel-images/ -n 8

Expected behavior Successfully transfer of full file.

Screenshots

image

Client Log client.log

Environment info (please complete the following information):

sarahwooders commented 1 year ago

Fixed by #757