search

skyplane-project / skyplane

🔥 Blazing fast bulk data transfers between any cloud 🔥
https://skyplane.org
Apache License 2.0
1.08k stars 62 forks source link

S3 us-east to us-west transfer failes #697

Closed DImuthuUpe closed 1 year ago

DImuthuUpe commented 1 year ago

Describe the bug I have a bucket with data in us-east-2 and I want to copy data in the bucket to a bucket in us-west-2. When I tried to do skyplane cp, I get the following error. I setup the key and secret using aws configure and then ran skyplane init before initiating the copy operation

⠴ Provisioning gateway instances ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/2 0:00:1601:05:44 [WARN] An error occurred (InvalidKeyPair.NotFound) when calling the RunInstances operation: The key pair 'skyplane-us-west-2' does not exist

DImuthuUpe commented 1 year ago

Fixed. I manually created the skyplane-us-west-2 key in us-west-2 and copied the .pem file to the local file path.

parasj commented 1 year ago

Thanks for noting this. I'm closing the issue but please reopen the issue if you encounter this again.

troycarlson commented 1 year ago

For anyone who stumbles upon this issue in the future, maybe my experience is helpful. I encountered the same error message (The key pair 'skyplane-us-west-2' does not exist) but under slightly different circumstances. After successfully running a skyplane job where the destination was a bucket in one AWS account, I tried running another skyplane job where the destination bucket was in another AWS account, which failed with the previously mentioned error. I simply removed the key pair that skyplane generated for the first job and let skyplane generate a new key pair for use in the new destination AWS account:

rm -rf ~/.skyplane/keys/aws`

Then ran skyplane again, it generated new keys, and everything worked.