search

skyplane-project / skyplane

🔥 Blazing fast bulk data transfers between any cloud 🔥
https://skyplane.org
Apache License 2.0
1k stars 58 forks source link

[request] Cost analysis not showing up when cp to local #734

Open mrnewguy opened 1 year ago

mrnewguy commented 1 year ago

Describe the bug

I was attempting to transfer data from Google Cloud -> local, and S3 -> local , and the cost analysis did not show up, unlike previously when I transferred data from GCS -> S3.

To Reproduce Steps to reproduce the behavior (please include the full Skyplane command you ran):

  1. Run skyplane cp S3_BUCKET_LOCATION . --recursive
  2. The files will start downloading straight away without giving a cost estimate.

Expected behavior

  1. The files should give a cost estimate before downloading the files (I think) .

Screenshots If applicable, add screenshots to help explain your problem.

telegram-cloud-photo-size-5-6196993859842585499-y

Transfer client log

For some reason the command did not show "Storing debug information for transfer in /tmp/skyplane/transfer_logs/..."

Environment info (please complete the following information):

Additional context Add any other context about the problem here.

sarahwooders commented 1 year ago

HI @mrnewguy - thanks for posting this! Skyplane actually does not fully support local transfers, so defaults to cloud CLIs in these cases (which do not provide cost estimates). Since this isn't a bug, I'll close this issue, but please join our Slack if you'd like to chat further or have feedback!

parasj commented 1 year ago

Ah @sarahwooders I asked @mrnewguy to post a feature request here so @ShishirPatil can add this to the upcoming on-prem release

ShishirPatil commented 1 year ago

For others who may come across this issue, Skyplane currently defaults to native cloud provider API to transfer files. We do not support parallel, and over-lay optimizations from Skyplane but are very close to doing so.