skyplane-project / skyplane

🔥 Blazing fast bulk data transfers between any cloud 🔥
https://skyplane.org
Apache License 2.0
1.08k stars 62 forks source link

[bug] Local transfers not working #823

Open sarahwooders opened 1 year ago

sarahwooders commented 1 year ago

Describe the bug Local transfers on Skyplane are not working, even though they should fall back to the AWS CLI.

 _____ _   ____   _______ _       ___   _   _  _____
/  ___| | / /\ \ / / ___ \ |     / _ \ | \ | ||  ___|
\ `--.| |/ /  \ V /| |_/ / |    / /_\ \|  \| || |__
 `--. \    \   \ / |  __/| |    |  _  || . ` ||  __|
/\__/ / |\  \  | | | |   | |____| | | || |\  || |___
\____/\_| \_/  \_/ \_|   \_____/\_| |_/\_| \_/\____/

Azure resource group and UMI name not configured correctly. Please reinit Azure with `skyplane init --reinit-azure`.
Using Skyplane version 0.3.1
Logging to: /tmp/skyplane/transfer_logs/20230502_011907-0d16be55/client.log
Will transfer objects from gcp:us-central1-a to aws:us-east-1
  VMs to provision: 1x aws:us-east-1, 1x gcp:us-central1-a
  Estimated egress cost: $0.12/GB
  /home/ubuntu/workspace/experiments/datasets/azure/azure_100/keys.json => s3://066278b49515470d93fc8f990d0a3d52/keys.json (7.02KB)
Continue? [Y/n]: Y
Transfer starting (Tip: Enable auto-confirmation with `skyplane config set autoconfirm true`)
01:19:11 [ERROR] Error running <lambda>: Cannot import googleapiclient.errors. Install skyplane with gcp support: `pip install
'skyplane'`
Uncaught exception: (ImportError) Cannot import googleapiclient.errors. Install skyplane with gcp support: `pip install 'skyplane'`
Please check the log file for more information, and ensure to include it if reporting an issue on Github.

To Reproduce Run a local->S3 transfer:

skyplane cp -r azure_100/ s3://066278b49515470d93fc8f990d0a3d52/

This will produce the above error.

Run a S3->local transfer:

skyplane cp s3://066278b49515470d93fc8f990d0a3d52/c5e49338a46a4391892c97fb6aa26528 .

This will silently fail.

Environment info (please complete the following information):

Additional context This error is likely due to bugs in the POSIXInterface.