skyplane-project / skyplane

🔥 Blazing fast bulk data transfers between any cloud 🔥
https://skyplane.org
Apache License 2.0
1k stars 58 forks source link

[Fix] Azure init issues #787

Closed zizhong closed 1 year ago

zizhong commented 1 year ago

Fixing several Azure init issues as follows.

  1. The warning message aborts the skyplane init process "--scope argument will become required for creating a role assignment in the breaking change release of the fall of 2023. Please explicitly specify --scope."
  2. ⠇ Provisioning VMs ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/1 0:00:2523:52:12 [ERROR] Error running _provision_task: 'eastus'
    Traceback (most recent call last):
    File "/home/zzz/workspace/skyplane/skyplane/cli/cli_transfer.py", line 382, in cp
    dp.provision(spinner=True)
    File "/home/zzz/workspace/skyplane/skyplane/api/dataplane.py", line 131, in provision
    uuids = self.provisioner.provision(
    File "/home/zzz/workspace/skyplane/skyplane/api/provisioner.py", line 205, in provision
    results: List[Tuple[ProvisionerTask, compute.Server]] = do_parallel(
    File "/home/zzz/workspace/skyplane/skyplane/utils/fn.py", line 57, in do_parallel
    args, result = future.result()
    File "/home/zzz/.pyenv/versions/3.8.10/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
    File "/home/zzz/.pyenv/versions/3.8.10/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
    File "/home/zzz/.pyenv/versions/3.8.10/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
    File "/home/zzz/workspace/skyplane/skyplane/utils/fn.py", line 43, in wrapped_fn
    return args, func(args)
    File "/home/zzz/workspace/skyplane/skyplane/api/provisioner.py", line 154, in _provision_task
    server = self.azure.provision_instance(task.region, task.vm_type, use_spot_instances=task.spot, tags=task.tags)
    File "/home/zzz/workspace/skyplane/skyplane/utils/imports.py", line 33, in wrapped
    return fn(*modules_imported, *args, **kwargs)
    File "/home/zzz/workspace/skyplane/skyplane/compute/azure/azure_cloud_provider.py", line 356, in provision_instance
    "hardware_profile": {"vm_size": self.lookup_valid_instance(location, vm_size)},
    File "/home/zzz/workspace/skyplane/skyplane/compute/azure/azure_cloud_provider.py", line 113, in lookup_valid_instance
    if instance_name in sku_mapping:
    KeyError: 'eastus'
zizhong commented 1 year ago

@parasj There is an issue in test-azure, saying "Cannot import azure.mgmt.quota.AzureQuotaExtensionAPI". Locally I installed azure-mgmt-quota and worked. However, the attempts of adding it to poetry failed where it seemed that poetry cannot resolve the version of package.

sarahwooders commented 1 year ago

Thanks for adding this fix @zizhong - the fix looks good to me, but I'll look into what's causing the Azure test to fail tomorrow.