Expected behavior
Files copied to GCP - I don't care about other cloud systems.
Screenshots
/ ___| | / /\ \ / / ___ \ | / _ \ | \ | || ___|
\ `--.| |/ / \ V /| |_/ / | / /_\ \| \| || |__
`--. \ \ \ / | __/| | | _ || . ` || __|
/\__/ / |\ \ | | | | | |____| | | || |\ || |___
\____/\_| \_/ \_/ \_| \_____/\_| |_/\_| \_/\____/
Azure resource group and UMI name not configured correctly. Please reinit Azure with `skyplane init --reinit-azure`.
Logging to: /tmp/skyplane/transfer_logs/20240508_072013-4b456028/client.log
Uncaught exception: (AssertionError) SCP Credentials not found
Please check the log file for more information, and ensure to include it if reporting an issue on Github.
Transfer client log
(parent paths trimmed, shouldn't be relevant)
07:20:13 [ERROR] Uncaught exception: AssertionError: SCP Credentials not found
07:20:13 [ERROR] Traceback:
Traceback (most recent call last):
File ".../skyplane/env/bin/skyplane", line 8, in <module>
sys.exit(app())
File ".../skyplane/env/lib/python3.10/site-packages/typer/main.py", line 326, in __call__
raise e
File ".../skyplane/env/lib/python3.10/site-packages/typer/main.py", line 309, in __call__
return get_command(self)(*args, **kwargs)
File ".../skyplane/env/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File ".../skyplane/env/lib/python3.10/site-packages/typer/core.py", line 723, in main
return _main(
File ".../skyplane/env/lib/python3.10/site-packages/typer/core.py", line 193, in _main
rv = self.invoke(ctx)
File ".../skyplane/env/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File ".../skyplane/env/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File ".../skyplane/env/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File ".../skyplane/env/lib/python3.10/site-packages/typer/main.py", line 692, in wrapper
return callback(**use_params)
File ".../skyplane/skyplane/cli/cli_transfer.py", line 471, in cp
return run_transfer(src, dst, recursive, debug, multipart, confirm, max_instances, max_connections, solver, "cp")
File ".../skyplane/skyplane/cli/cli_transfer.py", line 351, in run_transfer
cli = SkyplaneCLI(src_region_tag=src_region_tag, dst_region_tag=dst_region_tag, args=args)
File ".../skyplane/skyplane/cli/cli_transfer.py", line 67, in __init__
self.client = skyplane.SkyplaneClient(
File ".../skyplane/skyplane/api/client.py", line 65, in __init__
self.provisioner = Provisioner(
File ".../skyplane/skyplane/api/provisioner.py", line 77, in __init__
self._make_cloud_providers()
File ".../skyplane/skyplane/api/provisioner.py", line 92, in _make_cloud_providers
self.scp = compute.SCPCloudProvider(auth=self.scp_auth)
File ".../skyplane/skyplane/compute/scp/scp_cloud_provider.py", line 54, in __init__
self.network = network if network else SCPNetwork(self.auth)
File ".../skyplane/skyplane/compute/scp/scp_network.py", line 38, in __init__
self.scp_client = SCPClient()
File ".../skyplane/skyplane/compute/scp/scp_utils.py", line 117, in __init__
assert os.path.exists(self.credentials), "SCP Credentials not found"
AssertionError: SCP Credentials not found
Environment info (please complete the following information):
OS: Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Python version 3.10.12
Skyplane version: git head - 4602d9cec2083142831f7f208b413ca2bf091bd8
Additional context
When that problem was fixed (disabled) I ran into a problem with the scp load quota.
skyplane/planner/planner.py:53 Attempts to load and json parse the scp_quota file. My file (default init) exists but is empty, the json parse fails.
Workaround
The following changes allowed me to proceed by disabling SCP, they are a reasonable work around but they don't actually attempt to fix the problem properly.
diff --git a/skyplane/api/provisioner.py b/skyplane/api/provisioner.py
index 7e09817..d3a4048 100644
--- a/skyplane/api/provisioner.py
+++ b/skyplane/api/provisioner.py
@@ -89,7 +89,7 @@ class Provisioner:
self.azure = compute.AzureCloudProvider(auth=self.azure_auth)
self.gcp = compute.GCPCloudProvider(auth=self.gcp_auth)
self.ibmcloud = compute.IBMCloudProvider(auth=self.ibmcloud_auth)
- self.scp = compute.SCPCloudProvider(auth=self.scp_auth)
+ self.scp = None
def init_global(self, aws: bool = True, azure: bool = True, gcp: bool = True, ibmcloud: bool = True, scp: bool = True):
"""
diff --git a/skyplane/planner/planner.py b/skyplane/planner/planner.py
index dde2846..2e75ace 100644
--- a/skyplane/planner/planner.py
+++ b/skyplane/planner/planner.py
@@ -48,9 +48,6 @@ class Planner:
if os.path.exists(gcp_quota_path):
with gcp_quota_path.open("r") as f:
quota_limits["gcp"] = json.load(f)
- if os.path.exists(scp_quota_path):
- with scp_quota_path.open("r") as f:
- quota_limits["scp"] = json.load(f)
self.quota_limits = quota_limits
# Loading the vcpu information - a dictionary of dictionaries
Describe the bug Attempting to copy files from the local filesystem to a Google storage bucket.
To Reproduce
Expected behavior Files copied to GCP - I don't care about other cloud systems.
Screenshots
Transfer client log (parent paths trimmed, shouldn't be relevant)
Environment info (please complete the following information):
Additional context
When that problem was fixed (disabled) I ran into a problem with the scp load quota. skyplane/planner/planner.py:53 Attempts to load and json parse the scp_quota file. My file (default init) exists but is empty, the json parse fails.
Workaround
The following changes allowed me to proceed by disabling SCP, they are a reasonable work around but they don't actually attempt to fix the problem properly.